Abstract
We propose a method to reduce the computational cost and memory consumption of existing neural networks, by exploiting spatial redundancies in images. Our method dynamically splits the image into blocks and processes low-complexity regions at a lower resolution. Our novel BlockPad module, implemented in CUDA, replaces zero-padding in order to prevent the discontinuities at patch borders of which existing methods suffer, while keeping memory consumption under control. We demonstrate SegBlocks on Cityscapes semantic segmentation, where the number of floating point operations is reduced by 30% with only 0.2% loss in accuracy (mIoU), and an inference speedup of 50% is achieved with 0.7% decrease in mIoU.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: IEEE CVPR 2016 Proceedings, pp. 3213–3223 (2016)
Figurnov, M., et al.: Spatially adaptive computation time for residual networks. In: IEEE CVPR 2017 Proceedings, pp. 1039–1048 (2017)
Huang, Y.H., Proesmans, M., Georgoulis, S., Van Gool, L.: Uncertainty based model selection for fast semantic segmentation. In: MVA 2019 Proceedings, pp. 1–6 (2019)
Marin, D., et al.: Efficient segmentation: learning downsampling near semantic boundaries. In: IEEE CVPR 2019 Proceedings, pp. 2131–2141 (2019)
Orsic, M., Kreso, I., Bevandic, P., Segvic, S.: In defense of pre-trained imagenet architectures for real-time semantic segmentation of road-driving images. In: IEEE CVPR 2019 Proceedings, pp. 12607–12616 (2019)
Ren, M., Pokrovsky, A., Yang, B., Urtasun, R.: SBNet: sparse blocks network for fast inference. In: IEEE CVPR 2018 Proceedings, pp. 8711–8720 (2018)
Romera, E., Alvarez, J.M., Bergasa, L.M., Arroyo, R.: ERFNet: efficient residual factorized convnet for real-time semantic segmentation. IEEE Trans. Intell. Transp. Syst. 19(1), 263–272 (2017)
Verelst, T., Tuytelaars, T.: Dynamic convolutions: exploiting spatial sparsity for faster inference. In: IEEE CVPR 2020 Proceedings, pp. 2320–2329 (2020)
Wu, T., Lei, Z., Lin, B., Li, C., Qu, Y., Xie, Y.: Patch proposal network for fast semantic segmentation of high-resolution images. In: AAAI 2020 Proceedings, pp. 12402–12409 (2020)
Wu, Z., Shen, C., Hengel, A.V.D.: Real-time semantic image segmentation via spatial sparsity. arXiv preprint arXiv:1712.00213 (2017)
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 334–349. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_20
Acknowledgement
The work was funded by the HAPPY and CELSA-project.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Verelst, T., Tuytelaars, T. (2020). SegBlocks: Towards Block-Based Adaptive Resolution Networks for Fast Segmentation. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12539. Springer, Cham. https://doi.org/10.1007/978-3-030-68238-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-68238-5_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68237-8
Online ISBN: 978-3-030-68238-5
eBook Packages: Computer ScienceComputer Science (R0)