Pyramidal region context module for semantic segmentation
Authors: 
Tingting Liang, Qijie Zhao, Zhuoying Wang, Kaiyu Shan, Huan Zhang, Yongtao Wang, Zhi TangAuthors Info & Claims
Tingting Liang, Qijie Zhao, Zhuoying Wang, Kaiyu Shan, Huan Zhang, Yongtao Wang, Zhi TangAuthors Info & ClaimsArticle No.: 102, Pages 1 - 6
Abstract
Context modeling is widely exploited to enhance semantic correlation in semantic segmentation task. Recent approaches (e.g., OCNet, CCNet and DANet) apply non-local type of network to capture the context information. However, they are not accurate enough for handling scale-varying objects due to that they consider very little local dependencies of the adjacent pixels. In this work, we address the complex scene segmentation problem by combining region dependencies and global contextual information. Motivated by the fact that scale of objects largely varies on images, we design the Pyramidal Region Context Module(PRCM) to handle the neighbor relationship of multi-scale regions. In addition, we adopt a depth-to-space layer(PixelShuffle) to form the Scale Transfer Classifier (STC). Based on the two newly proposed modules, we introduce an end-to-end segmentation network - Pyramidal Region Network(PRNet). We empirically demonstrate the effectiveness of our approach on Cityscapes dataset, the results have shown impressive improvement compared with baselines. Notably, PRNet obtains mean IoU of 81.3 on test set of Cityscapes.
References
[1]
Liang-Chieh Chen and George Papandreou et al. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation. CoRR (2017).
[2]
Liang-Chieh Chen and George Papandreou et al. 2018. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. (2018).
[3]
Liang-Chieh Chen and Yukun Zhu et al. 2018. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In ECCV.
[4]
Marius et al. Cordts. 2016. The Cityscapes Dataset for Semantic Urban Scene Understanding. In CVPR.
[5]
J. Deng and Dong et al. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR.
[6]
Jun Fu and Jing Liu et al. 2018. Dual Attention Network for Scene Segmentation. CoRR (2018).
[7]
Golnaz Ghiasi and Charless C. Fowlkes. 2016. Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation. In ECCV.
[8]
Kaiming He and Xiangyu Zhang et al. 2016. Deep Residual Learning for Image Recognition. In CVPR.
[9]
Han Hu and Jiayuan Gu et al. 2018. Relation Networks for Object Detection. In CVPR.
[10]
Zilong Huang and Xinggang Wang et al. 2018. CCNet: Criss-Cross Attention for Semantic Segmentation. CoRR (2018).
[11]
Guosheng Lin and Anton Milan et al. 2017. RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation. In CVPR.
[12]
Guosheng Lin and Chunhua Shen et al. 2016. Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation. In CVPR.
[13]
Yun Liu and Ming-Ming Cheng et al. 2017. Richer Convolutional Features for Edge Detection. In CVPR.
[14]
Jonathan Long and Evan Shelhamer et al. 2015. Fully convolutional networks for semantic segmentation. In CVPR.
[15]
Wenzhe Shi and Jose Caballero et al. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In CVPR.
[16]
Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR (2014).
[17]
Christian Szegedy and Wei Liu et al. 2015. Going deeper with convolutions. In CVPR.
[18]
Xiaolong Wang and Ross B. Girshick et al. 2018. Non-Local Neural Networks. In CVPR.
[19]
Saining Xie and Zhuowen Tu. 2017. Holistically-Nested Edge Detection. International Journal of Computer Vision (2017).
[20]
Jimei Yang and Brian L. Price et al. 2016. Object Contour Detection with a Fully Convolutional Encoder-Decoder Network. In CVPR.
[21]
Maoke Yang and Kun Yu et al. 2018. DenseASPP for Semantic Segmentation in Street Scenes. In CVPR.
[22]
Changqian Yu and Jingbo Wang et al. 2018. Learning a Discriminative Feature Network for Semantic Segmentation. In CVPR.
[23]
Zhiding Yu and Chen Feng et al. 2017. CASENet: Deep Category-Aware Semantic Edge Detection. In CVPR.
[24]
Yuhui Yuan and Jingdong Wang. 2018. OCNet: Object Context Network for Scene Parsing. CoRR (2018).
[25]
Hang Zhang and Kristin J. Dana et al. 2018. Context Encoding for Semantic Segmentation. In CVPR.
[26]
Hengshuang Zhao and Jianping Shi et al. 2017. Pyramid Scene Parsing Network. In CVPR.
[27]
Peng Zhou and Bingbing Ni et al. 2018. Scale-Transferrable Object Detection. In CVPR.
Index Terms
- Pyramidal region context module for semantic segmentation
Recommendations
Pyramid Self-attention for Semantic Segmentation
Pattern Recognition and Computer VisionAbstractSelf-attention is vital in computer vision since it is the building block of Transformer and can model long-range context for visual recognition. However, computing pairwise self-attention between all pixels for dense prediction tasks (e.g., ...
Self-attention neural architecture search for semantic image segmentation
AbstractSelf-attention can capture long-distance dependencies and is widely used in semantic segmentation. Existing methods mainly use two kinds of self-attentions, i.e., spatial attention and channel attention, which can capture the relations in H W ...
Information entropy induced graph convolutional network for semantic segmentation
AbstractSemantic segmentation is a challenging task in computer vision. Contextual information has been proven to be helpful for this task. But few previous works have ever tied to explicitly utilizing contextual information from easy regions in images ...
Highlights- Using entropy to determine easy and hard image regions for semantic segmentation.
- Apply graph convolution to utilize easy regions to assist in recognizing hard regions.
- Model the entropy utilization process as a plug-and-play ...
Comments
Information & Contributors
Information
Published In
May 2019
963 pages
ISBN:9781450371582
DOI:10.1145/3321408
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 May 2019
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ACM TURC 2019
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 108Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in