Abstract
Semantic segmentation of remote sensing images is of great significance to the interpretation of remote sensing images. Recently, convolutional neural networks have been increasingly used in this task since it can effectively learn the features in the image. In this paper, an end-to-end semantic segmentation framework, Multi-scale Spatial Aggregation Network (MSAN), is proposed for the remote sensing image segmentation. At first, a classical SegNet is employed as the backbone of the network because its simple structure is suitable for the remote sensing images that have a small quantity of samples. Then several skip connections and a densely connected block are utilized to enhance the usage of the low-level feature and reduce the loss of the detail information in the original image. Moreover, multi-scale spatial information fusion module and a spatial path are added between the encoder and decoder of SegNet, which can effectively extract the features of objects with different sizes in the remote sensing images. Finally, a smoothing algorithm is presented to improve the blocking effect of the remote sensing image segmentation results. The proposed MSAN is tested on the ISPRS Vaihingen dataset and the dataset of a city in southern China, which obtains the satisfactory results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gupta, S., Arbelaez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from rgb-d images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 564–571 (2013)
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33715-4_54
Yu, C., Wang, J., Gao, C., Yu, G., Shen, C., Sang, N.: Context prior for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2020)
Lin, G., Milan, A., Shen, C., Reid, I.: Refinenet: multi-path refinement networks with identity mappings for high-resolution semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Fu, J., Liu, J., Tian, H., Fang, Z., Lu, H.: Dual attention network for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
Silva-RodrÃguez, J., Colomer, A., Naranjo, V.: WeGleNet: a weakly-supervised convolutional neural network for the semantic segmentation of gleason grades in prostate histology images. Computeriz. Med. Imag. Graph. 88, 101846 (2021). https://doi.org/10.1016/j.compmedimag.2020.101846
Zhang, Z., Huang, J., Jiang, T., et al.: Semantic segmentation of very high-resolution remote sensing image based on multiple band combinations and patchwise scene analysis. J. Appl. Remote Sens. 14(1), 1 (2020)
Ren, X., Bo, L., Fox, D.: Rgb-(d) scene labeling: features and algorithms. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 2759–2766. IEEE (2012)
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Learning a discriminative feature network for semantic segmentation. In. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Paszke, A., Chaurasia, A., Kim, S., Culurciello, E.: Enet: A deep neural network architecture for real-time semantic segmentation. (2016)
Liu, J., Geng, Y., Zhao, J., et al.: Image semantic segmentation use multiple-threshold probabilistic R-CNN with feature fusion. Symmetry 13(2), 207 (2021)
Bergum, S., Saad, A., Stahl, A.: Automatic in-situ instance and semantic segmentation of planktonic organisms using Mask R-CNN[C]. In: IEEE Oceanic Engineering Society & Marine Technology Society. IEEE (2020)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Int. 39(4), 640–651 (2014)
Chen, G., Zhang, X., Wang, Q.: Symmetrical dense-shortcut deep fully convolutional networks for semantic segmentation of very-high-resolution remote sensing images. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 11(5), 1633–1644 (2018)
Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Int. 39(12), 2481–2495 (2017)
Huang, G., Liu, Z., Laurens, V.D.M., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 334–349. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_20
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representation (2014)
Chen, L.C., Papandreou, G., Kokkinos, I.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Int. 40(4), 834–848 (2018)
Chen, L.C.: Rethinking atrous convolution for semantic image segmentation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2017)
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 6230–6239 (2017)
Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., Hajishirzi, H.: Espnet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 561–580. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_34
Zhang, H., et al.: Context encoding for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7151–7160 (2018)
Li, X., Liu, Z., Luo, P., Loy, C.C., Tang, X.: Not all pixels are equal: difficulty-aware semantic segmentation via deep layer cascade. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Mehta, S., Rastegari, M., Shapiro, L.G., Hajishirzi, H.: Espnetv2: a light-weight, power efficient, and general purpose convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
Audebert, N., Le Saux, B., Lefèvre, S.: Semantic segmentation of earth observation data using multimodal and multi-scale deep networks. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 180–196. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54181-5_12
He, Y., Dong, X., Kang, G., et al.: Asymptotic soft filter pruning for deep convolutional neural networks. IEEE Trans. Cybern. 50(8), 3594–3604 (2020)
Chen, G., Zhang, X., Wang, Q.: Symmetrical dense-shortcut deep fully convolutional networks for semantic segmentation of very-high-resolution remote sensing images. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 11(5), 1633–1644 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)
Bo, Y., Lu, Y., Fang, C.: Semantic segmentation for high spatial resolution remote sensing images based on convolution neural network and pyramid pooling module. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 1–10 (2018)
Liu, Y., Fan, B., Wang, L.: Semantic labeling in very high resolution images via a self-cascaded convolutional neural network. ISPRS J. Photogramm. Remote. Sens. 145, 78–95 (2018)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the International Conference on Machine Learning, ICML, pp. 448–456 (2015)
Nair, V., Hinton, G.E.: Rectifified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (2010)
Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Proc. Int. J. Comput. Vis. 111(1), 98–136 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Sun, X., Gu, J., Feng, J., Yang, S., Jiao, L. (2022). Multi-scale Spatial Aggregation Network for Remote Sensing Image Segmentation. In: Shi, Z., Jin, Y., Zhang, X. (eds) Intelligence Science IV. ICIS 2022. IFIP Advances in Information and Communication Technology, vol 659. Springer, Cham. https://doi.org/10.1007/978-3-031-14903-0_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-14903-0_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14902-3
Online ISBN: 978-3-031-14903-0
eBook Packages: Computer ScienceComputer Science (R0)
