skip to main content
10.1145/3321408.3322635acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesacm-turcConference Proceedingsconference-collections
research-article

Pyramidal region context module for semantic segmentation

Published: 17 May 2019 Publication History

Abstract

Context modeling is widely exploited to enhance semantic correlation in semantic segmentation task. Recent approaches (e.g., OCNet, CCNet and DANet) apply non-local type of network to capture the context information. However, they are not accurate enough for handling scale-varying objects due to that they consider very little local dependencies of the adjacent pixels. In this work, we address the complex scene segmentation problem by combining region dependencies and global contextual information. Motivated by the fact that scale of objects largely varies on images, we design the Pyramidal Region Context Module(PRCM) to handle the neighbor relationship of multi-scale regions. In addition, we adopt a depth-to-space layer(PixelShuffle) to form the Scale Transfer Classifier (STC). Based on the two newly proposed modules, we introduce an end-to-end segmentation network - Pyramidal Region Network(PRNet). We empirically demonstrate the effectiveness of our approach on Cityscapes dataset, the results have shown impressive improvement compared with baselines. Notably, PRNet obtains mean IoU of 81.3 on test set of Cityscapes.

References

[1]
Liang-Chieh Chen and George Papandreou et al. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation. CoRR (2017).
[2]
Liang-Chieh Chen and George Papandreou et al. 2018. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. (2018).
[3]
Liang-Chieh Chen and Yukun Zhu et al. 2018. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In ECCV.
[4]
Marius et al. Cordts. 2016. The Cityscapes Dataset for Semantic Urban Scene Understanding. In CVPR.
[5]
J. Deng and Dong et al. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR.
[6]
Jun Fu and Jing Liu et al. 2018. Dual Attention Network for Scene Segmentation. CoRR (2018).
[7]
Golnaz Ghiasi and Charless C. Fowlkes. 2016. Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation. In ECCV.
[8]
Kaiming He and Xiangyu Zhang et al. 2016. Deep Residual Learning for Image Recognition. In CVPR.
[9]
Han Hu and Jiayuan Gu et al. 2018. Relation Networks for Object Detection. In CVPR.
[10]
Zilong Huang and Xinggang Wang et al. 2018. CCNet: Criss-Cross Attention for Semantic Segmentation. CoRR (2018).
[11]
Guosheng Lin and Anton Milan et al. 2017. RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation. In CVPR.
[12]
Guosheng Lin and Chunhua Shen et al. 2016. Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation. In CVPR.
[13]
Yun Liu and Ming-Ming Cheng et al. 2017. Richer Convolutional Features for Edge Detection. In CVPR.
[14]
Jonathan Long and Evan Shelhamer et al. 2015. Fully convolutional networks for semantic segmentation. In CVPR.
[15]
Wenzhe Shi and Jose Caballero et al. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In CVPR.
[16]
Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR (2014).
[17]
Christian Szegedy and Wei Liu et al. 2015. Going deeper with convolutions. In CVPR.
[18]
Xiaolong Wang and Ross B. Girshick et al. 2018. Non-Local Neural Networks. In CVPR.
[19]
Saining Xie and Zhuowen Tu. 2017. Holistically-Nested Edge Detection. International Journal of Computer Vision (2017).
[20]
Jimei Yang and Brian L. Price et al. 2016. Object Contour Detection with a Fully Convolutional Encoder-Decoder Network. In CVPR.
[21]
Maoke Yang and Kun Yu et al. 2018. DenseASPP for Semantic Segmentation in Street Scenes. In CVPR.
[22]
Changqian Yu and Jingbo Wang et al. 2018. Learning a Discriminative Feature Network for Semantic Segmentation. In CVPR.
[23]
Zhiding Yu and Chen Feng et al. 2017. CASENet: Deep Category-Aware Semantic Edge Detection. In CVPR.
[24]
Yuhui Yuan and Jingdong Wang. 2018. OCNet: Object Context Network for Scene Parsing. CoRR (2018).
[25]
Hang Zhang and Kristin J. Dana et al. 2018. Context Encoding for Semantic Segmentation. In CVPR.
[26]
Hengshuang Zhao and Jianping Shi et al. 2017. Pyramid Scene Parsing Network. In CVPR.
[27]
Peng Zhou and Bingbing Ni et al. 2018. Scale-Transferrable Object Detection. In CVPR.

Index Terms

  1. Pyramidal region context module for semantic segmentation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ACM TURC '19: Proceedings of the ACM Turing Celebration Conference - China
    May 2019
    963 pages
    ISBN:9781450371582
    DOI:10.1145/3321408
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 17 May 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. neural networks
    2. self-attention
    3. semantic segmentation

    Qualifiers

    • Research-article

    Conference

    ACM TURC 2019

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 108
      Total Downloads
    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 25 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media