Abstract
Remarkable progress has been made in real-time semantic segmentation by leveraging lightweight backbone networks and auxiliary low-level training tasks. Despite several techniques have been proposed to mitigate accuracy degradation resulting from model reduction, challenging regions often exhibit substantial uncertainty values in segmentation results. To tackle this issue, we propose an effective structure named Uncertainty-aware Boundary Attention Network(UBANet). Specifically, we model the segmentation uncertainty via prediction variance during training and involve it as a regularization item into optimization objective to improve segmentation performance. Moreover, we employ uncertainty maps to investigate the role of low-level supervision in segmentation task. And we reveal that directly fusing high- and low-level features leads to the overshadowing of large-scale low-level features by the encompassing local contexts, thus hindering the synergy between the segmentation task and low-level tasks. To address this issue, we design a Low-level Guided Feature Fusion Module that avoids the direct fusion of high- and low-level features and instead employs low-level features as guidance for the fusion of multi-scale contexts. Extensive experiments demonstrate the efficiency and effectiveness of our proposed method by achieving the state-of-the-art latency-accuracy trade-off on Cityscapes and CamVid benchmark.
This work was supported by National Key R&D Program of China under Grant No.2021ZD0114600, National Natural Science Foundation of China (62006230, 62206283, 62076235).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: ECCV (1), pp. 44–57 (2008)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. arXiv:1606.00915 (2016)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: ECCV (2018)
Chen, W., Gong, X., Liu, X., Zhang, Q., Li, Y., Wang, Z.: Fasterseg: searching for faster real-time semantic segmentation. In: ICLR (2020)
Cheng, B., Schwing, A.G., Kirillov, A.: Per-pixel classification is not all you need for semantic segmentation (2021)
Contributors, M.: Mmsegmentation: openmmlab semantic segmentation toolbox and benchmark (2020). https://github.com/open-mmlab/mmsegmentation
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)
Fan, M., et al.: Rethinking bisenet for real-time semantic segmentation. In: CVPR, pp. 9716–9725 (June 2021)
Fu, J., et al.: Dual attention network for scene segmentation. In: CVPR (2019)
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: Representing model uncertainty in deep learning (2016)
Goan, E., Fookes, C.: Uncertainty in real-time semantic segmentation on embedded systems. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 4490–4500 (June 2023)
Jin, Y., Han, D., Ko, H.: Trseg: trransformer for semantic segmentation. Pattern Recogn. Lett. 148, 29–35 (2021)
Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision? (2017)
Li, H., Xiong, P., Fan, H., Sun, J.: Dfanet: deep feature aggregation for real-time semantic segmentation (2019). 10.48550/ARXIV.1904.02216, https://arxiv.org/abs/1904.02216
Li, X.,et al.: Improving semantic segmentation via decoupled body and edge supervision. In: ECCV (2020)
Li, X., et al.: Semantic flow for fast and accurate scene parsing. In: ECCV (2020)
Li, X., Zhou, Y., Pan, Z., Feng, J.: Partial order pruning: for best speed/accuracy trade-off in neural architecture search. CoRR abs/1903.03777 (2019). https://arxiv.org/abs/1903.03777
Mukhoti, J., van Amersfoort, J., Torr, P.H., Gal, Y.: Deep deterministic uncertainty for semantic segmentation. arXiv preprint arXiv:2111.00079 (2021)
Pan, H., Hong, Y., Sun, W., Jia, Y.: Deep dual-resolution networks for real-time and accurate semantic segmentation of traffic scenes. IEEE Trans. Intell. Transp. Syst. 24(3), 3448–3460 (2022)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vision 128(2), 336–359 (oct 2019). https://doi.org/10.1007/s11263-019-01228-7, https://doi.org/10.10072Fs11263-019-01228-7
Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE TPAMI 39(4), 640–651 (2017)
Wang, J., et al.: Rtformer: efficient design for real-time semantic segmentation with transformer. In: NeurIPS (2022)
Wang, L., Li, D., Zhu, Y., Tian, L., Shan, Y.: Dual super-resolution learning for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2020)
Wang, Y., Peng, J., Zhang, Z.: Uncertainty-aware pseudo label refinery for domain adaptive semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9092–9101 (2021)
Xu, J., Xiong, Z., Bhattacharyya, S.P.: Pidnet: a real-time semantic segmentation network inspired from pid controller (2022)
Yu, C., Gao, C., Wang, J., Yu, G., Shen, C., Sang, N.: Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. In: IJCV, pp. 1–18 (2021)
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Bisenet: bilateral segmentation network for real-time semantic segmentation. In: ECCV, pp. 325–341 (2018)
Yuan, Y., Wang, J.: Ocnet: object context network for scene parsing. CoRR abs/1809.00916 (2018)
Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: Icnet for real-time semantic segmentation on high-resolution images. In: ECCV, pp. 405–420 (2018)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: CVPR (2017)
Zheng, Z., Yang, Y.: Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. Int. J. Comput. Vision 129(4), 1106–1120 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhu, Y., Zhu, B., Chen, Y., Wang, J. (2024). Uncertainty-Aware Boundary Attention Network for Real-Time Semantic Segmentation. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14427. Springer, Singapore. https://doi.org/10.1007/978-981-99-8435-0_31
Download citation
DOI: https://doi.org/10.1007/978-981-99-8435-0_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8434-3
Online ISBN: 978-981-99-8435-0
eBook Packages: Computer ScienceComputer Science (R0)