Skip to main content

Advertisement

SpringerLink
Log in

A hybrid-attention semantic segmentation network for remote sensing interpretation in land-use surveillance

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Remote sensing interpretation for surveillance of land use often needs to mark out construction disturbance on satellite imagery, such as illegal buildings or spoil area. These disturbance region annotated by a set of surveillance rules contain the corresponding image characteristics which are regarded as semantic information in computer vision. Different from the natural Landscapes interpretation, the semantic information of construciton disturbance region shows more complex to extract with lack of available training dataset and interference of the various sizes of targets. This paper proposes a hybrid attention semantic segmentation network (HAssNet) which can extract the target and its surroundings through a large receptive field for multi-scale targets. Based on the full convolutional networks (FCN), spatial attention mechanism is firstly introduced to acquire the position of segmentation target with the global correlations, so that the small targets in large scale scene are guaranteed not to be omitted in semantic features extraction. Secondly, channel attention mechanism is designed to assign higher weights to task-related channels for semantic consistency. Experimental results on an open remote sensing dataset show that HAssNet achieves average 6.7% improvement in mIoU than the state-of-the-art segmentation networks. In a land use surveillance project, HAssNet shows considerable performance compared with manual interpretation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://naic.pcl.ac.cn/frame/2.

References

  1. Lillesand T, Kiefer RW, Chipman J (2015) Remote sensing and image interpretation. Wiley, Hoboken

    Google Scholar 

  2. Lv N, Chen C, Qiu T, Sangaiah AK (2018) Deep learning and superpixel feature extraction based on contractive autoencoder for change detection in sar images. IEEE Trans Industr Inf 14(12):5530–5538

    Article  Google Scholar 

  3. Cong W, Chen C, Qingqi P, Zhiyuan J, Shugong X (2021) An information centric in-network caching scheme for 5g-enabled internet of connected vehicles. IEEE Trans Mob Comput. https://doi.org/10.1109/TMC.2021.3137219

    Article  Google Scholar 

  4. Lin F, Wang J, Sun Yun (2016) Establishment of interpretation signs of disturbance patterns in production and construction projects based on high resolution remote sensing images. Soil Water Conserv China 11:16–20

    Google Scholar 

  5. Strahler AH (1980) The use of prior probabilities in maximum likelihood classification of remotely sensed data. Remote Sens Environ 10(2):135–163

    Article  Google Scholar 

  6. Goldberg M, Shlien S (1978) A clustering scheme for multispectral images. IEEE Trans Syst Man Cybern 8(2):86–92

    Article  Google Scholar 

  7. Xu Y, Xie Z, Feng Y, Chen Z (2018) Road extraction from high-resolution remote sensing imagery using deep learning. Remote Sens 10(9):1461

    Article  Google Scholar 

  8. Yu K, Liu J, Lu Z (2017) An adaptive multi-threshold image segmentation algorithm based on object-oriented classification for high-resolution remote sensing images. In: AOPC 2017: Optical sensing and imaging technology and applications, international society for optics and photonics. vol 10462. p 104624B

  9. Ali M, Clausi D (2001) Using the canny edge detector for feature extraction and enhancement of remote sensing images. In: IGARSS 2001. Scanning the present and resolving the future. Proceedings. IEEE 2001 international geoscience and remote sensing symposium (Cat. No.01CH37217), vol 5, pp 2298–2300. https://doi.org/10.1109/IGARSS.2001.977981

  10. Ming D, Li J, Wang J, Zhang M (2015) Scale parameter selection by spatial statistics for Geobia: using mean-shift based multi-scale segmentation as an example. ISPRS J Photogramm Remote Sens 106:28–41

    Article  Google Scholar 

  11. Zhang G, Jia X, Hu J (2015) Superpixel-based graphical model for remote sensing image mapping. IEEE Trans Geosci Remote Sens 53(11):5861–5871

    Article  Google Scholar 

  12. Wang Y, Li D, Wang Y (2019) Realization of remote sensing image segmentation based on k-means clustering. In: IOP Conference Series: Materials Science and Engineering, IOP Publishing, vol 490, p 072008

  13. Liu T, Wen XB, Quan JJ, Xu XQ (2008) Multiscale sar image segmentation using support vector machines. In: 2008 Congress on Image and Signal Processing, IEEE, vol 3, pp 706–709

  14. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  15. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3431–3440

  16. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  17. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  18. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognitio. pp 2881–2890

  19. Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587

  20. Chen C, Liu L, Wan S, Hui X, Pei Q (2021) Data dissemination for industry 4.0 applications in internet of vehicles based on short-term traffic prediction. ACM Trans Internet Technol (TOIT) 22(1):1–18

    Article  Google Scholar 

  21. Byeon W, Breuel TM, Raue F, Liwicki M (2015) Scene labeling with LSTM recurrent neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 3547–3555

  22. Sukhbaatar S, Szlam A, Weston J, Fergus R (2015) End-to-end memory networks. arXiv preprint arXiv:1503.08895

  23. Woo S, Park J, Lee JY, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  24. Tian C, Xu Y, Li Z, Zuo W, Fei L, Liu H (2020) Attention-guided CNN for image denoising. Neural Netw 124:117–129

    Article  Google Scholar 

  25. Li J, Fan D, Yang L, Gu S, Lu G, Xu Y, Zhang D (2021) Layer-output guided complementary attention learning for image defocus blur detection. IEEE Trans Image Process 30:3748–3763. https://doi.org/10.1109/TIP.2021.3065171

    Article  Google Scholar 

  26. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems. pp 5998–6008

  27. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7794–7803

  28. Cao Y, Xu J, Lin S, Wei F, Hu H (2019) Gcnet: non-local networks meet squeeze-excitation networks and beyond. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops

  29. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 3146–3154

  30. Li Y, Chen X, Zhu Z, Xie L, Huang G, Du D, Wang X (2019) Attention-guided unified network for panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 7026–7035

  31. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7132–7141

  32. Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) Eca-net: efficient channel attention for deep convolutional neural networks, 2020 IEEE. In: CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE

  33. Wang X, Cai Z, Gao D, Vasconcelos N (2019) Towards universal object detection by domain attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  34. Luo H, Chen C, Fang L, Zhu X, Lu L (2019) High-resolution aerial images semantic segmentation using deep fully convolutional network with channel attention mechanism. IEEE J Select Top Appl Earth Observ Remote Sens 12(9):3492–3507

    Article  Google Scholar 

  35. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778

  36. Huang Z, Wang X, Huang L, Huang C, Wei Y, Liu W (2019) Ccnet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp 603–612

  37. Zhong Z, Lin ZQ, Bidart R, Hu X, Daya IB, Li Z, Zheng WS, Li J, Wong A (2020) Squeeze-and-attention networks for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 13065–13074

  38. Yuan Y, Huang L, Guo J, Zhang C, Chen X, Wang J (2018) Ocnet: object context network for scene parsing. arXiv preprint. arXiv:1809.00916

  39. Zhu X, Cheng D, Zhang Z, Lin S, Dai J (2019) An empirical study of spatial attention mechanisms in deep networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp 6688–6697

  40. Chen C, Jiange J, Rufei F, Lanlan C, Cong L, Shaohua W (2021) An intelligent caching strategy considering time-space characteristics in vehicular named data networks. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2021.3128012

    Article  Google Scholar 

  41. Niu R, Sun X, Tian Y, Diao W, Chen K, Fu K (2021) Hybrid multiple attention network for semantic segmentation in aerial images. IEEE Trans Geosci Remote Sens 2021(60):1–18

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (2019YFE0196600), the National Natural Science Foundation of China (62072360, 61902292, 62001357, 62072359, 62072355), the key research and development plan of Shaanxi province (2021ZDLGY02-09, 2019ZDLGY13-07, 2019ZDLGY13-04, 2020JQ-844), the key laboratory of embedded system and service computing (Tongji University) (ESSCKF2019-05), Ministry of Education, the Xi’an Science and Technology Plan (20RGZN0005) and the Xi’an Key Laboratory of Mobile Edge Computing and Security (201805052-ZD3CG36).

Author information

Authors and Affiliations

  1. School of Electronic Engineering, Xidian University, Xi’an, 710071, China

    Ning Lv, Zenghui Zhang & Jiaxuan Deng

  2. State Grid JiLin Province Electric Power Company Limited Information Communication Company, Changchun, 130000, China

    Cong Li

  3. The National Laboratory of Radar Signal Processing, Xidian University, Xi’an, 710071, China

    Tao Su

  4. The State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an, 710071, China

    Chen Chen

  5. Ministry of Water Resources of China, Beijing, 100053, China

    Yang Zhou

Authors
  1. Ning Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zenghui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiaxuan Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tao Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lv, N., Zhang, Z., Li, C. et al. A hybrid-attention semantic segmentation network for remote sensing interpretation in land-use surveillance. Int. J. Mach. Learn. & Cyber. 14, 395–406 (2023). https://doi.org/10.1007/s13042-022-01517-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01517-7

Keywords

Search

Navigation