Skip to main content
SpringerLink
Log in

MFGAN: multi feature guided aggregation network for remote sensing image

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Remote sensing image change detection is an important technology in remote sensing data analysis. Existing mainstream solutions are divided into supervised and unsupervised solutions. Among supervised methods, most remote sensing image change methods based on deep learning are related to semantic segmentation. However, these methods only use deep learning model to extract and process single-level information of the image, and do not separately train the changed and unchanged areas according to the task characteristics of change detection. Also this model do not aggregate the deep and shallow-level feature information, so many local change details cannot be detected. This paper proposes a multilateral feature-guided aggregation network. The network firstly extracts the whole information, the changed area information and the unchanged area information of remote sensing image through the main network and two auxiliary networks: difference network and assimilation network. Then, the whole information of the image is aggregated with the changed and unchanged region information by feature aggregation network (all subnetworks are composed of convolutional neural networks (CNNs)). The proposed method is end-to-end trainable, and each component in the network does not need to be trained separately.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Tison C, Nicolas J-M, Tupin F, Maître H (2004) A new statistical model for Markovian classification of urban areas in high-resolution SAR images. IEEE Trans Geosci Remote Sens 42(10):2046–2057

    Article  Google Scholar 

  2. Sommer S, Hill J, Megier J (1998) The potential of remote sensing for monitoring rural land use changes and their effects on soil conditions. Agric Ecosyst Environ 67(2–3):197–209

    Article  Google Scholar 

  3. Fichera CR, Modica G, Pollino M (2012) Land cover classification and change-detection analysis using multi-temporal remote sensed imagery and landscape metrics. Eur J Remote Sens 45(1):1–18

    Article  Google Scholar 

  4. Gillespie TW, Chu J, Frankenberg E, Thomas D (2007) Assessment and prediction of natural hazards from satellite imagery. Prog Phys Geogr 31(5):459–470

    Article  Google Scholar 

  5. Dong L, Shan J (2013) A comprehensive review of earthquake-induced building damage detection with remote sensing techniques. ISPRS J of Photogramm Remote Sens 84:85–99

    Article  Google Scholar 

  6. Keim Daniel A (2000) Designing pixel-oriented visualization techniques: theory and applications. IEEE Trans Vis Comput Graph 6(1):59–78

    Article  Google Scholar 

  7. Coad P, Yourdon E (1991) Object oriented analysis

  8. Blasco J, Aleixos N, Moltó E (2007) Computer vision detection of peel defects in citrus by means of a region oriented segmentation algorithm. J Food Eng 81(3):535–543

    Article  Google Scholar 

  9. Bovolo F, Bruzzone L, Solano-Correa YT (2017) Multitemporal analysis of remotely sensed image data

  10. Saha S, Solano-Correa YT, Bovolo F, Bruzzone L (2020) Unsupervised deep transfer learning-based change detection for HR multispectral images. IEEE Geosci Remote Sens Lett 18:856–860

    Article  Google Scholar 

  11. Peng D, Zhang Y, Guan H (2019) End-to-end change detection for high resolution satellite images using improved UNet++. Remote Sens 11(11):1382

    Article  Google Scholar 

  12. Ball JE, Anderson DT, Chan CS Sr (2017) Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community. J Appl Remote Sens 11(4):042609

    Article  Google Scholar 

  13. Nielsen AA, Conradsen K, Simpson JJ (1998) Multivariate alteration detection (MAD) and MAF postprocessing in multispectral, bitemporal image data: new approaches to change detection studies. Remote Sens Environ 64(1):1–19

    Article  Google Scholar 

  14. Nielsen AA (2007) The regularized iteratively reweighted mad method for change detection in multi-and hyperspectral data. IEEE Trans Image Process 16(2):463–478

    Article  MathSciNet  Google Scholar 

  15. Zhang H, Gong M, Zhang P, Su L, Shi J (2016) Feature-level change detection using deep representation and feature change analysis for multispectral imagery. IEEE Geosci Remote Sens Lett 13(11):1666–1670

    Article  Google Scholar 

  16. Malila William A (1980) Change vector analysis: an approach for detecting forest changes with landsat. In: LARS symposia, p 385

  17. Sesnie SE, Gessler PE, Finegan B, Thessler S (2008) Integrating landsat TM and SRTM-DEM derived variables with decision trees for habitat classification and change detection in complex neotropical environments. Remote Sens Environ 112(5):2145–2159

    Article  Google Scholar 

  18. Chen B, Xia M, Huang J (2021) Mfanet: a multi-level feature aggregation network for semantic segmentation of land cover. Remote Sens 13(4):731

    Article  Google Scholar 

  19. Xia M, Wang T, Zhang Y, Liu J, Xu Y (2021) Cloud/shadow segmentation based on global attention feature fusion residual network for remote sensing imagery. Int J Remote Sens 42(6):2022–2045

    Article  Google Scholar 

  20. Xia M, Cui Y, Zhang Y, Xu Y, Liu J, Xu Y (2021) DAU-Net: a novel water areas segmentation structure for remote sensing image. Int J Remote Sens 42(7):2594–2621

    Article  Google Scholar 

  21. Xia M, Wang K, Song W, Chen C, Li Y et al (2020) Non-intrusive load disaggregation based on composite deep long short-term memory network. Expert Syst Appl 160:113669

    Article  Google Scholar 

  22. Gong M, Zhao J, Liu J, Miao Q, Jiao L (2015) Change detection in synthetic aperture radar images based on deep neural networks. IEEE Trans Neural Netw Learn Syst 27(1):125–138

    Article  MathSciNet  Google Scholar 

  23. Zhan Y, Fu K, Yan M, Sun X, Wang H, Qiu X (2017) Change detection based on deep siamese convolutional network for optical aerial images. IEEE Geosci Remote Sens Lett 14(10):1845–1849

    Article  Google Scholar 

  24. Al-Huda Z, Peng B, Yang Y, Algburi RN, Ahmad M, Khurshid F, Moghalles K (2021) Weakly supervised semantic segmentation by iteratively refining optimal segmentation with deep cues guidance. Neural Comput Appl 1–26

  25. Zhang Y, Li X, Lin M, Chiu B, Zhao M (2020) Deep-recursive residual network for image semantic segmentation. Neural Comput Appl 32(16):12935–12947

    Article  Google Scholar 

  26. Ouni A, Royer E, Chevaldonné M, Dhome M (2021) Leveraging semantic segmentation for hybrid image retrieval methods. Neural Comput Appl 1–19

  27. Moujahid A, Dornaika F, Ruichek Y, Hammoudi K (2019) Towards semantic segmentation of orthophoto images using graph-based community identification. Neural Comput Appl 31(2):1155–1163

    Article  Google Scholar 

  28. Zhang L, Sheng Z, Li Y, Sun Q, Zhao Y, Feng D (2019) Image object detection and semantic segmentation based on convolutional neural network. Neural Comput Appl 1–10

  29. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation, pp 3431–3440

  30. Varghese A, Gubbi J, Ramaswamy A, Balamuralidhar P (2018) Changenet: a deep learning architecture for visual change detection. In:Proceedings of the European conference on computer vision (ECCV) workshops, pp 1–6

  31. Peng D, Bruzzone L, Zhang Y, Guan H, Ding H, Huang X (2020) Semicdnet: a semisupervised convolutional neural network for change detection in high resolution remote-sensing images. IEEE Trans Geosci Remote Sens 59:5891–5906

    Article  Google Scholar 

  32. Chen H, Shi Z (2020) A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sens 12(10):1662

    Article  Google Scholar 

  33. Xia M, Zhang X, Weng L, Xu Y et al (2020) Multi-stage feature constraints learning for age estimation. IEEE Trans Inf Forensics Secur 15:2417–2428

    Article  Google Scholar 

  34. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  35. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprintarXiv:1409.1556

  36. Gao H, Liu Z, Van Der Maaten L, Weinberger Kilian Q (2017) Densely connected convolutional networks. In:Proceedings of the IEEE conference on computer vision and pattern recognition, pp4700–4708

  37. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In:Proceedings of the IEEE conference on computer vision and pattern recognition, pp770–778

  38. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In:Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–1258

  39. Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In:Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pp855–864

  40. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  41. Paszke A, Chaurasia A, Kim S, Culurciello E (2016) Enet: a deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147

  42. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In:International conference on machine learning, pp 448–456. PMLR

  43. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, pp 5–323

  44. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  45. Yang K, Xia GS, Liu Z, Du B, Yang W, Pelillo M (2020) Asymmetric siamese networks for semantic change detection. arXiv preprint arXiv:2010.05687

  46. Daudt RC, Le Saux B, Boulch A, Gousseau Y (2019) Multitask learning for large-scale semantic change detection. Comput Visi Image Underst 187:102783

    Article  Google Scholar 

  47. Allison GB, Barnes CJ (1983) Estimation of evaporation from non-vegetated surfaces using natural deuterium. Nature 301(5896):143–145

    Article  Google Scholar 

  48. Tong XY, Xia GS, Lu Q, Shen H, Li S, You S, Zhang L (2020) Land-cover classification with high-resolution remote sensing images using transferable deep models. Remote Sens Environ 237:111322

    Article  Google Scholar 

  49. Long Y, Xia GS, Li S, Yang W, Yang MY, Zhu XX, Zhang L, Li D (2020) Dirs: on creating benchmark datasets for remote sensing image interpretation. arXiv preprint arXiv:2006.12485

  50. Xia GS, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) Dota: a large-scale dataset for object detection in aerial images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp3974–3983

  51. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pp 801–818

  52. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In International conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  53. Yu C, Wang J, Gao C, Yu G, Shen C, Sang N (2018) Bisenet: Bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European conference on computer vision (ECCV), pp 325–341

  54. Chen J, Yuan Z, Peng J, Chen L, Huang H, Zhu J, Liu Y, Li H (2020) Dasnet: dual attentive fully convolutional siamese networks for change detection in high-resolution satellite images. IEEE J Sel Top Appl Earth Obs Remote Sens 14:1194–1206

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of PR China (42075130).

Author information

Authors and Affiliations

  1. Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Shengguang Chu, Peng Li & Min Xia

  2. Jiangsu Key Laboratory of Big Data Analysis Technology, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Min Xia

Authors
  1. Shengguang Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chu, S., Li, P. & Xia, M. MFGAN: multi feature guided aggregation network for remote sensing image. Neural Comput & Applic 34, 10157–10173 (2022). https://doi.org/10.1007/s00521-022-06999-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-06999-8

Keywords

Search

Navigation