Skip to main content

Advertisement

SpringerLink
Log in

Bio-inspired Multi-Sensory Pathway Network for Change Detection

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Change detection is a significantly important task in the field of remote sensing and can be widely used in the urban construction planning, disaster survey, resource management, etc. Previous studies have shown that most of the state-of-the-art change detection methods are based on the deep learning networks. However, the problem of change detection still cannot be effectively solved due to the variations in illumination, resolution, quality, scale, location, etc. The robust methods for change detection need to be further investigated. This study aimed to address the insufficient robustness problem of the change detection between the multi-temporal images. The biological mechanism of parallel processing architecture in visual pathways gives us an inspiration to design a sensory framework with multi-sensory pathways. We propose a new framework named multi-sensory pathway network (MSPN). This framework is inspired by the parallel processing mechanism of the human visual information. Specifically, the framework utilizes three diverse but related sensory pathways: sensory pathway-1, sensory pathway-2, and sensory pathway-3. The three sensory pathways of the proposed framework are not simply parallel processing, but with some related connections. The sensory pathway-1 adopts the early fusion strategy to learn the changed information. The sensory pathway-2 uses the middle concatenation strategy to learn the changed information, while the sensory pathway-3 utilizes the middle difference strategy to learn the change information. Two fusion strategies, namely average fusion and maximum fusion, are designed for the framework. The experimental datasets consists of BCDD, LEVIR-CD, and CDD. Four metrics including overall accuracy (OA), precision, recall, and F1 are used to evaluate the competitive algorithms. The primary metric is F1. The proposed method, respectively, achieves the best F1 scores with 84.55%, 88.14%, and 85.11% on the three experimental datasets. The quantitative ablation results show the effectiveness of multi-sensory pathways on the BCDD, LEVIR-CD, and CDD. The qualitative ablation results demonstrate that different sensory pathways perform different perception mechanisms, though they belong to the united framework. The comprehensive results of MSPN-AF on the BCDD and MSPN-MF on the LEVIR-CD and CDD are superior to other methods. The experimental results demonstrate the effectiveness and robustness of the proposed method, both qualitatively and quantitatively. The proposed MSPN can promote the technology exploration of the bionic and explainable neural network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. https://study.rsgis.whu.edu.cn/pages/download/building_dataset.html

  2. https://justchenhao.github.io/LEVIR/

  3. https://drive.google.com/file/d/1GX656JqqOyBi_Ef0w65kDGVto-nHrNs9

References

  1. Dou A, Wang X, Yuan X, Wang S. The loss assessment method of building earthquake damage using the remote sensing and building grid data. In: Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS); 2016. p. 4255-8.

  2. Ye X, Liu M, Wang J, Qin Q, Ren H, Wang J, et al. Building-based damage detection from postquake image using multiple-feature analysis. IEEE Geosci Remote Sens Lett. 2017;14(4):499–503.

    Article  Google Scholar 

  3. Sun L, Tang Y, Zhang L. Rural building detection in high-resolution imagery based on a two-stage cnn model. IEEE Geosci Remote Sens Lett. 2017;14(11):1998–2002.

    Article  Google Scholar 

  4. Li X, Chen M, Nie F, Wang Q. A multiview-based parameter free framework for group detection. In: Proc. Conference on Artificial Intelligence; 2017. p. 4147-53.

  5. Zhao ZZ, Wang HT, Wang C, Wang ST, Li YQ. Fusing lidar data and aerial imagery for building detection using a vegetation-mask-based connected filter. IEEE Geosci Remote Sens Lett. 2019;16(8):1299–303.

    Article  Google Scholar 

  6. Wang Q, Liu S, Chanussot J, Li X. Scene classification with recurrent attention of vhr remote sensing images. IEEE Geosci Remote Sens Lett. 2019;57(2):1155–67.

    Article  Google Scholar 

  7. Nie Y, Zeng Q, Zhang H, Wang Q. Building damage detection based on opce matching algorithm using a single post-event polsar data. Remote Sens. 2021;13(6):1146.

    Article  Google Scholar 

  8. Konstantinidis D, Stathaki T, Argyriou V, Grammalidis N. Building detection using enhanced hog-lbp features and region refinement processes. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2017;10(3):888–905.

    Article  Google Scholar 

  9. Yan Z, Huazhong R, Desheng C. The research of building earthquake damage object-oriented change detection based on ensemble classifier with remote sensing image. In: Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS); 2018. p. 4950-3.

  10. Chen B, Deng L, Duan Y, Huang S, Zhou J. Building change detection based on 3D reconstruction. In: Proc. IEEE International Conference on Image Processing (ICIP); 2015. p. 4126-30.

  11. Benedek C, Descombes X, Zerubia J. Building development monitoring in multitemporal remotely sensed image pairs with stochastic birth-death dynamics. IEEE Trans Pattern Anal Mach Intell. 2012;34(1):33–50.

    Article  Google Scholar 

  12. Huang J, Xia G, Hu F, Zhang L. Accurate building detection in vhr remote sensing images using geometric saliency. In: Proc. IEEE International Geoscience and Remote Sensing Symposium (IGARSS); 2018. p. 3991-4.

  13. Barnes CF, Fritz H, Yoo J. Hurricane disaster assessments with image-driven data mining in high-resolution satellite imagery. IEEE Trans Geosci Remote Sens. 2007;45(6–1):1631–40.

    Article  Google Scholar 

  14. Bazi Y, Melgani F, Al-Sharari HD. Unsupervised change detection in multispectral remotely sensed imagery with level set methods. IEEE Trans Geosci Remote Sens. 2010;48(8):3178–87.

    Article  Google Scholar 

  15. Bouziani M, Goieta K, He DC. Automatic change detection of buildings in urban environment from very high spatial resolution images using existing geodatabase and prior knowledge. ISPRS J Photogramm Remote Sens. 2010;65(1):143–53.

    Article  Google Scholar 

  16. Bovolo F, Bruzzone L, Marconcini M. A novel approach to unsupervised change detection based on a semisupervised svm and a similarity measure. IEEE Trans Geosci Remote Sens. 2008;46(7):2070–82.

    Article  Google Scholar 

  17. Bruzzone L, Fernández-Prieto D. Automatic analysis of the difference image for unsupervised change detection. IEEE Trans Geosci Remote Sens 2000;38(3):1171-82.

  18. Chen X, Jin C, Shi Y, Yamaguchi Y. An automated approach for updating land cover maps based on integrated change detection and classification methods. ISPRS J Photogramm Remote Sens 2012;71(Jul.):86-95.

  19. Gong M, Zhao J, Liu J, Miao Q, Jiao L. Change detection in synthetic aperture radar images based on deep neural networks. IEEE Trans Neural Netw Learn Syst. 2016;27(1):125–38.

    Article  MathSciNet  Google Scholar 

  20. Du S, Zhang Y, Qin R, Yang Z, Zou Z, Tang Y, et al. Building change detection using old aerial images and new lidar data. Remote Sens. 2016;8(12).

  21. Chen G, Hay GJ, St-Onge B. A GEOBIA framework to estimate forest parameters from lidar transects, Quickbird imagery and machine learning: A case study in Quebec, Canada. Int J Appl Earth Obs Geoinf. 2012;15:28–37.

    Google Scholar 

  22. Wu C, Zhang F, Xia J, Xu Y, Li G, Xie J, et al. Building damage detection using u-net with attention mechanism from pre- and post-disaster remote sensing datasets. Remote Sens. 2021;13(5):905.

    Article  Google Scholar 

  23. Liu Y, Pang C, Zhan Z, Zhang X, Yang X. Building change detection for remote sensing images using a dual-task constrained deep siamese convolutional network model. IEEE Geosci Remote Sens Lett. 2021;18(5):811–5.

    Article  Google Scholar 

  24. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). vol. 9351; 2015. p. 234-41.

  25. Song K, Jiang J. AGCDetNet: An attention-guided network for building change detection in high-resolution remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2021;14:4816–31.

    Article  Google Scholar 

  26. Lebedev M, Vizilter Y, Vygolov O, Knyaz V, Rubis A. Change detection in remote sensing images using conditional adversarial networks. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. 2018;XLII-2:565-71.

  27. Ji SP, Wei SQ, Lu M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set. IEEE Trans Geosci Remote Sens. 2019;57(1):574–86.

    Article  Google Scholar 

  28. Daudt RC, Saux BL, Boulch A. fully convolutional siamese networks for change detection. In: Proc. IEEE International Conference on Image Processing (ICIP); 2018. p. 4063-7.

  29. Daudt RC, Saux BL, Boulch A, Gousseau Y. Multitask learning for large-scale semantic change detection. Comput Vis Image Underst. 2019;187.

  30. Chen H, Shi Z. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sens. 2020;12(10):1662.

    Article  Google Scholar 

  31. Liu J, Gong M, Qin AK, Zhang P. A deep convolutional coupling network for change detection based on heterogeneous optical and radar images. IEEE Trans Neural Netw Learn Syst. 2018;29(3):545–59.

    Article  MathSciNet  Google Scholar 

  32. Dong H, Ma W, Wu Y, Gong M, Jiao L. Local descriptor learning for change detection in synthetic aperture radar images via convolutional neural networks. IEEE Access. 2019;7:15389–403.

    Article  Google Scholar 

  33. Yue Z, Gao F, Xiong Q, Wang J, Huang T, Yang E, et al. A novel semi-supervised convolutional neural network method for synthetic aperture radar image recognition. Cogn Comput. 2019.

  34. Gao F, Dong J, Li B, Xu Q. Automatic change detection in synthetic aperture radar images based on pcanet. IEEE Geosci Remote Sens Lett. 2016;13(12):1792–6.

    Article  Google Scholar 

  35. Li Y, Peng C, Chen Y, Jiao L, Zhou L, Shang R. A deep learning method for change detection in synthetic aperture radar images. IEEE Trans Geosci Remote Sens. 2019;57(8):5751–63.

    Article  Google Scholar 

  36. Qin R, Huang X, Gruen A, Schmitt G. Object-based 3-d building change detection on multitemporal stereo images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2015;8(5):2125–37.

    Article  Google Scholar 

  37. Xiao PF, Yuan M, Zhang XL, Feng XZ, Guo YW. Cosegmentation for object-based building change detection from high-resolution remotely sensed images. IEEE Trans Geosci Remote Sens. 2017;55(3):1587–603.

    Article  Google Scholar 

  38. Janalipour M, Mohammadzadeh A. Building damage detection using object-based image analysis and anfis from high-resolution image (case study: bam earthquake, Iran). IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2016;9(5):1937–45.

    Article  Google Scholar 

  39. Addink EA, Coillie FV, Jong SM. Introduction to the geobia 2010 special issue: from pixels to geographic objects in remote sensing image analysis. Int J Appl Earth Obs Geoinf. 2012;15:1–6.

    Google Scholar 

  40. Aguirre-Gutiérrez J, Seijmonsbergen AC, Duivenvoorden JF. Optimizing land cover classification accuracy for change detection, a combined pixel-based and object-based approach in a mountainous area in Mexico. Appl Geogr. 2012;34(1):29-37.

  41. Blaschke T. Object based image analysis for remote sensing. ISPRS J Photogramm Remote Sens. 2010;65(1):2–16.

    Article  Google Scholar 

  42. Bontemps S, Bogaert P, Titeux N, Defourny P. An object-based change detection method accounting for temporal dependences in time series with medium to coarse spatial resolution. Remote Sens Environ. 2008;112(6):3181–91.

    Article  Google Scholar 

  43. Huo C, Zhou Z, Lu H, Pan C, Chen K. Fast object-level change detection for vhr images. IEEE Geosci Remote Sens Lett. 2010;7(1):118–22.

    Article  Google Scholar 

  44. Li C, Dai H. Statistical object-based method for forest change detection using high-resolution remote sensing images. Sci Silvae Sin. 2017.

  45. Listner C, Niemeyer I. Object-based change detection. Photogrammetrie - Fernerkundung - Geoinformation. 2011;4(4):233–45.

    Article  Google Scholar 

  46. Yang X, Liu H, Gao X. Land cover changed object detection in remote sensing data with medium spatial resolution. Int J Appl Earth Obs Geoinf. 2015;38:129–37.

    Google Scholar 

  47. Kandel E, Schwartz J, Jessell T, Siegelbaum S, Hudspeth A. Principle of neural science. 2013.

  48. Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In: Proc. IEEE Computer Society; 2005. p. 539-46.

Download references

Funding

This work is supported by The National Natural Science Foundation of China (No. 61871470).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, 710072, Xi’an, China

    Kang Liu & Xuelong Li

Authors
  1. Kang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuelong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuelong Li.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, K., Li, X. Bio-inspired Multi-Sensory Pathway Network for Change Detection. Cogn Comput 14, 1421–1434 (2022). https://doi.org/10.1007/s12559-021-09968-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-021-09968-w

Keywords

Search

Navigation