Abstract
As a pixel-level prediction task, semantic segmentation need rich spatial information. However, most popular real-time network architectures tend to compromise spatial resolution to increase speed, but the accuracy is greatly reduced as a result. Therefore, we propose a novel Feature Enhancement Module (FEM) to extracted and enhanced the future map before the image down-sample on the backbone. Meanwhile, since the low-layer have rich detail information and high-level contain more semantic information, we propose a Feature Extraction and Fusion Module (FEFM) to fuse low-level and high-level feature representation. Based on the FEM and FEFM, we introduce a real-time semantic segmentation network FEENET. Experiments on Cityscapes and CamVid datasets demonstrate that the proposed FEENET achieves a balance between speed computation and accuracy. Without additional processing and pre-training, it achieves 75.47% Mean IoU on the Cityscapes test dataset with only 4.35G Flops and a speed of 94 FPS on a single RTX 2080Ti card. Code is avilable at https://github.com/favoMJ/FEENet.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Fu, J., Liu, J., Tian, H., et al.: Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)
Huang, Z., Wang, X., Wei, Y., et al.: CCNet: criss-cross attention for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2020)
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)
Treml, M., et al.: Speeding up semantic segmentation for autonomous driving. In: MLITS, NIPS Workshop (2016)
Wu, Z., Shen, C., van den Hengel, A.: Real-time semantic image segmentation via spatial sparsity. arXiv preprint arXiv:1712.00213 (2017)
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 334–349. Springer, Cham (2018a). https://doi.org/10.1007/978-3-030-01261-8_20
Wang, L.W., Siu, W.C., Liu, Z.S., et al.: Deep relighting networks for image light source manipulation. arXiv preprint arXiv:2008.08298 (2020)
Li, H., Xiong, P., Fan, H., Sun, J.: DFANet: deep feature aggregation for real-time semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 4, p. 13 (2019b)
Wang, P., Chen, P., Yuan, Y., et al.: Understanding convolution for semantic segmentation. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1451–1460. IEEE (2018)
Mehta, S., Rastegari, M., Shapiro, L.G. Hajishirzi, H.: ESPNetv2: a light-weight, power efficient, and general purpose convolutional neural network. CoRR, abs/1811.11431 (2018)
Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for real-time semantic segmentation on high-resolution images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 418–434. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_25
Paszke, A., Chaurasia, A., Kim, S., et al.: ENet: a deep neural network architecture for real-time semantic segmentation (2016)
Lin, G., Milan, A., Shen, C., et al.: RefineNet: multi-path refinement networks for high-resolution semantic segmentation (2016)
Si, H., Zhang, Z., Lv, F., et al.: Real-time semantic segmentation via multiply spatial fusion network. arXiv preprint arXiv:1911.07217 (2019)
Huang, Z., Wang, X., Huang, L., et al.: CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 603–612 (2019)
Zou, X., Xiao, F., Yu, Z., et al.: Delving deeper into anti-aliasing in ConvNets. arXiv preprint arXiv:2008.09604 (2020)
Zhang, R.: Making convolutional networks shift-invariant again. arXiv preprint arXiv:1904.11486 (2019)
Wu, T., Tang, S., Zhang, R., et al.: CGNet: a light-weight context guided network for semantic segmentation. arXiv preprint arXiv:1811.08201 (2018)
Romera, E., Alvarez, J.M., Bergasa, L.M., et al.: ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans. Intell. Transp. Syst. 19(1), 263–272 (2017)
Wang, Y., Zhou, Q., Liu, J., et al.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation (2019)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision & Pattern Recognition. IEEE Computer Society (2016)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. arXiv e-prints (2016)
Lee, Y., Hwang, J., Lee, S., et al.: An energy and GPU-computation efficient backbone network for real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of International Conference on Machine Learning (ICML), pp. 807–814 (2010)
Ma, N., Zhang, X., Huang, J., et al.: WeightNet: revisiting the design space of weight networks. arXiv preprint arXiv:2007.11823 (2020)
Cordts, M., Omran, M., Ramos, S., et al.: The cityscapes dataset for semantic urban scene understanding (2016)
Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 44–57. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88682-2_5
Li, G., Yun, I., Kim, J., et al.: DABNet: depth-wise asymmetric bottleneck for real-time semantic segmentation (2019)
Yu, C., Wang, J., Peng, C., et al.: Learning a discriminative feature network for semantic segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2018)
Chen, L.C., Papandreou, G., Kokkinos, I., et al.: DeepLab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)
Wang, Y., Zhou, Q., Liu, J., et al.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1860–1864. IEEE (2019)
Zhang, B., Li, W., Hui, Y., et al.: MFENet: multi-level feature enhancement network for real-time semantic segmentation. Neurocomputing (2020)
Zhuang, J., Yang, J., Gu, L., et al.: ShelfNet for fast semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (No. U1603115), National key R&D plan project (2017YF C0820702-3) and National Engineering Laboratory for Public Safety Risk Perception and Control by Big Data (XJ201810101).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tan, S., Yang, W., Lin, J., Yu, W. (2021). FEENET: A Real-Time Semantic Segmentation via Feature Extraction and Enhancement. In: Ning, L., Chau, V., Lau, F. (eds) Parallel Architectures, Algorithms and Programming. PAAP 2020. Communications in Computer and Information Science, vol 1362. Springer, Singapore. https://doi.org/10.1007/978-981-16-0010-4_9
Download citation
DOI: https://doi.org/10.1007/978-981-16-0010-4_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0009-8
Online ISBN: 978-981-16-0010-4
eBook Packages: Computer ScienceComputer Science (R0)