Skip to main content

FEENET: A Real-Time Semantic Segmentation via Feature Extraction and Enhancement

  • Conference paper
  • First Online:
Parallel Architectures, Algorithms and Programming (PAAP 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1362))

  • 815 Accesses

Abstract

As a pixel-level prediction task, semantic segmentation need rich spatial information. However, most popular real-time network architectures tend to compromise spatial resolution to increase speed, but the accuracy is greatly reduced as a result. Therefore, we propose a novel Feature Enhancement Module (FEM) to extracted and enhanced the future map before the image down-sample on the backbone. Meanwhile, since the low-layer have rich detail information and high-level contain more semantic information, we propose a Feature Extraction and Fusion Module (FEFM) to fuse low-level and high-level feature representation. Based on the FEM and FEFM, we introduce a real-time semantic segmentation network FEENET. Experiments on Cityscapes and CamVid datasets demonstrate that the proposed FEENET achieves a balance between speed computation and accuracy. Without additional processing and pre-training, it achieves 75.47% Mean IoU on the Cityscapes test dataset with only 4.35G Flops and a speed of 94 FPS on a single RTX 2080Ti card. Code is avilable at https://github.com/favoMJ/FEENet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  2. Fu, J., Liu, J., Tian, H., et al.: Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)

    Google Scholar 

  3. Huang, Z., Wang, X., Wei, Y., et al.: CCNet: criss-cross attention for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2020)

    Google Scholar 

  4. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017)

    Article  Google Scholar 

  5. Treml, M., et al.: Speeding up semantic segmentation for autonomous driving. In: MLITS, NIPS Workshop (2016)

    Google Scholar 

  6. Wu, Z., Shen, C., van den Hengel, A.: Real-time semantic image segmentation via spatial sparsity. arXiv preprint arXiv:1712.00213 (2017)

  7. Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 334–349. Springer, Cham (2018a). https://doi.org/10.1007/978-3-030-01261-8_20

    Chapter  Google Scholar 

  8. Wang, L.W., Siu, W.C., Liu, Z.S., et al.: Deep relighting networks for image light source manipulation. arXiv preprint arXiv:2008.08298 (2020)

  9. Li, H., Xiong, P., Fan, H., Sun, J.: DFANet: deep feature aggregation for real-time semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 4, p. 13 (2019b)

    Google Scholar 

  10. Wang, P., Chen, P., Yuan, Y., et al.: Understanding convolution for semantic segmentation. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1451–1460. IEEE (2018)

    Google Scholar 

  11. Mehta, S., Rastegari, M., Shapiro, L.G. Hajishirzi, H.: ESPNetv2: a light-weight, power efficient, and general purpose convolutional neural network. CoRR, abs/1811.11431 (2018)

    Google Scholar 

  12. Zhao, H., Qi, X., Shen, X., Shi, J., Jia, J.: ICNet for real-time semantic segmentation on high-resolution images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 418–434. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_25

    Chapter  Google Scholar 

  13. Paszke, A., Chaurasia, A., Kim, S., et al.: ENet: a deep neural network architecture for real-time semantic segmentation (2016)

    Google Scholar 

  14. Lin, G., Milan, A., Shen, C., et al.: RefineNet: multi-path refinement networks for high-resolution semantic segmentation (2016)

    Google Scholar 

  15. Si, H., Zhang, Z., Lv, F., et al.: Real-time semantic segmentation via multiply spatial fusion network. arXiv preprint arXiv:1911.07217 (2019)

  16. Huang, Z., Wang, X., Huang, L., et al.: CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 603–612 (2019)

    Google Scholar 

  17. Zou, X., Xiao, F., Yu, Z., et al.: Delving deeper into anti-aliasing in ConvNets. arXiv preprint arXiv:2008.09604 (2020)

  18. Zhang, R.: Making convolutional networks shift-invariant again. arXiv preprint arXiv:1904.11486 (2019)

  19. Wu, T., Tang, S., Zhang, R., et al.: CGNet: a light-weight context guided network for semantic segmentation. arXiv preprint arXiv:1811.08201 (2018)

  20. Romera, E., Alvarez, J.M., Bergasa, L.M., et al.: ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans. Intell. Transp. Syst. 19(1), 263–272 (2017)

    Article  Google Scholar 

  21. Wang, Y., Zhou, Q., Liu, J., et al.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation (2019)

    Google Scholar 

  22. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision & Pattern Recognition. IEEE Computer Society (2016)

    Google Scholar 

  23. Chollet, F.: Xception: deep learning with depthwise separable convolutions. arXiv e-prints (2016)

    Google Scholar 

  24. Lee, Y., Hwang, J., Lee, S., et al.: An energy and GPU-computation efficient backbone network for real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)

    Google Scholar 

  25. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015)

    Google Scholar 

  26. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of International Conference on Machine Learning (ICML), pp. 807–814 (2010)

    Google Scholar 

  27. Ma, N., Zhang, X., Huang, J., et al.: WeightNet: revisiting the design space of weight networks. arXiv preprint arXiv:2007.11823 (2020)

  28. Cordts, M., Omran, M., Ramos, S., et al.: The cityscapes dataset for semantic urban scene understanding (2016)

    Google Scholar 

  29. Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 44–57. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88682-2_5

    Chapter  Google Scholar 

  30. Li, G., Yun, I., Kim, J., et al.: DABNet: depth-wise asymmetric bottleneck for real-time semantic segmentation (2019)

    Google Scholar 

  31. Yu, C., Wang, J., Peng, C., et al.: Learning a discriminative feature network for semantic segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2018)

    Google Scholar 

  32. Chen, L.C., Papandreou, G., Kokkinos, I., et al.: DeepLab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)

    Article  Google Scholar 

  33. Wang, Y., Zhou, Q., Liu, J., et al.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1860–1864. IEEE (2019)

    Google Scholar 

  34. Zhang, B., Li, W., Hui, Y., et al.: MFENet: multi-level feature enhancement network for real-time semantic segmentation. Neurocomputing (2020)

    Google Scholar 

  35. Zhuang, J., Yang, J., Gu, L., et al.: ShelfNet for fast semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No. U1603115), National key R&D plan project (2017YF C0820702-3) and National Engineering Laboratory for Public Safety Risk Perception and Control by Big Data (XJ201810101).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wenzhong Yang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tan, S., Yang, W., Lin, J., Yu, W. (2021). FEENET: A Real-Time Semantic Segmentation via Feature Extraction and Enhancement. In: Ning, L., Chau, V., Lau, F. (eds) Parallel Architectures, Algorithms and Programming. PAAP 2020. Communications in Computer and Information Science, vol 1362. Springer, Singapore. https://doi.org/10.1007/978-981-16-0010-4_9

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-0010-4_9

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-0009-8

  • Online ISBN: 978-981-16-0010-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics