Skip to main content

Low-sample size remote sensing image recognition based on a multihead attention integration network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

For a long time, small sample recognition for hyperspectral images has been a popular research topic. It is very difficult for an algorithm to simultaneously satisfy the requirements of feature mining, feature selection and feature integration. The traditional single model has difficulty completing multiple tasks at the same time, ultimately leading to poor small sample recognition results for remote sensing images. This paper proposes a multimodel joint algorithm for deep feature mining based on multiscale convolution (MC) under multihead attention (MA) and deep feature integration based on bidirectional independent recurrent neural networks (BiIndRNNs), MACBINet. First, this paper proposes a multihead attention mechanism that assigns multiple weight coefficients to each feature to better select remote sensing image features; then, it implements the deep mining of features and the retention of multiple deep features through multiscale convolution. Subsequently, it implements contextual semantic information integration for long-sequence features through bidirectional independent recurrent neural networks to avoid the problem of gradient disappearance during training on a small sample of data. Finally, the softmax function is used to perform recognition on three public remote sensing data sets. The experimental results prove that our proposed MACBINet achieves the best results to date for small sample classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Achour S, Chikr Elmezouar M, Taleb N, Kpalma K., Ronsin J. (2020). A PCA-PD fusion method for change detection in remote sensing multi temporal images[J]. Geocarto Int : 1–18

  2. Akhlaq MLM, Winarso G (2020). Comparative analysis of object-based and pixel-based classification of high-resolution remote sensing images for mapping coral reef geomorphic zones[C]//1st Borobudur international symposium on humanities, economics and social sciences (BIS-HESS 2019). Atlantis Press: 992–996

  3. Anders K, Winiwarter L, Lindenbergh R, Williams JG, Vos SE, Höfle B (2020) 4D objects-by-change: spatiotemporal segmentation of geomorphic surface change from LiDAR time series[J]. ISPRS J Photogramm Remote Sens 159:352–363

    Article  Google Scholar 

  4. Bakhti K, Arabi MEA, Chaib S et al (2020) Bi-directional LSTM model for classification of vegetation from satellite time series[C]//2020 Mediterranean and middle-east geoscience and remote sensing symposium (M2GARSS). IEEE:160–163

  5. Bioucas-Dias JM, Plaza A, Camps-Valls G, Scheunders P, Nasrabadi N, Chanussot J (2013) Hyperspectral remote sensing data analysis and future challenges[J]. IEEE Geoscience and remote sensing magazine 1(2):6–36

    Article  Google Scholar 

  6. Cai W, Wei Z (2020) PiiGAN: generative adversarial networks for pluralistic image Inpainting. IEEE Access 8:48451–48463

    Article  Google Scholar 

  7. Chen J, Chen S, Zhou P, Qian Y (2019) Deep neural network based Hyperspectral pixel classification with factorized spectral-spatial feature representation. IEEE Access 7:81407–81418

    Article  Google Scholar 

  8. Chen C, He X, Chu Y et al (2020) A new remote sensing image fusion method combining principal component analysis and curvelet transform[J]. MS&E 780(3):032054

    Google Scholar 

  9. Chen T, Zhao Y, Guo Y (2020) Sparsity-regularized feature selection for multi-class remote sensing image classification. Neural Comput & Applic 32:6513–6521

    Article  Google Scholar 

  10. Cui J, Zhang X, Wang W, Wang L (2020) Integration of optical and SAR remote sensing images for crop-type mapping based on a novel object-oriented feature selection method[J]. International Journal of Agricultural and Biological Engineering 13(1):178–190

    Article  Google Scholar 

  11. Deep Learning in Computer Vision: Principles and Applications[M]. CRC Press, 2020.

  12. Feng J, Wu X, Chen J, et al. (2019). “Joint multilayer spatial-spectral Classi-fication of Hyperspectral images based on CNN and Convlstm,” IEEE International Geoscience and Remote Sensing Symposium IEEE: 588–591

  13. Gao H, Yang Y, Yao D, Li C (2019) Hyperspectral image classification with pre-activation residual attention network[J]. IEEE Access 7:176587–176599

    Article  Google Scholar 

  14. Ghaffari R, Golpardaz M, Helfroush MS, Danyali H (2020) A fast, weighted CRF algorithm based on a two-step superpixel generation for SAR image segmentation[J]. Int J Remote Sens 41(9):3535–3557

    Article  Google Scholar 

  15. Huang M, Chen Q, Wang H (2020) A multivariable optical remote sensing image feature discretization method applied to marine vessel targets recognition[J]. Multimed Tools Appl 79(7):4597–4618

    Article  Google Scholar 

  16. Ji C, Ye M, Lu H, et al. (2019). Feature Extraction of Hyperspectral Imagery Based on Deep NMF[C]//IGARSS 2019-2019 IEEE international geoscience and remote sensing symposium. IEEE: 1092–1095

  17. LeCun Y, Bengio Y, Hinton G (2015) Deep learning[J]. Nature 521(7553):436–444

    Article  Google Scholar 

  18. Lee H, Kwon H (2017) Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing 26(10):4843–4855

    Article  MathSciNet  Google Scholar 

  19. Lei, L. X., & Peng, L. (2020). Training strategy of CNN for remote sensing image classification with active learning. May, 52(1)

  20. Li J, Bioucas-Dias JM, Plaza A (2012) Semisupervised hyperspectral image classification using soft sparse multinomial logistic regression[J]. IEEE Geosci Remote Sens Lett 10(2):318–322

    Google Scholar 

  21. Li Y, Fang S, Jiao L, Liu R, Shang R (2020) A multi-level attention model for remote sensing image captions[J]. Remote Sens 12(6):939

    Article  Google Scholar 

  22. Li P, Han L, Tao X, et al. (2020). Hashing nets for hashing: a quantized deep learning to hash framework for remote sensing image retrieval[J]. IEEE Trans Geosci Remote Sens

  23. Li G, Zhang C, Lei R, Zhang X, Ye Z, Li X (2020) Hyperspectral remote sensing image classification using three-dimensional-squeeze-and-excitation-DenseNet (3D-SE-DenseNet)[J]. Remote Sensing Letters 11(2):195–203

    Article  Google Scholar 

  24. Liu X, Zhou Y, Zhao J, Yao R, Liu B, Ma D, Zheng Y (2020) Multiobjective ResNet pruning by means of EMOAs for remote sensing scene classification[J]. Neurocomputing 381:298–305

    Article  Google Scholar 

  25. Pan E, Mei X, Wang Q, Ma Y, Ma J (2020) Spectral-spatial classification for hyperspectral image based on a single GRU[J]. Neurocomputing 387:150–160

    Article  Google Scholar 

  26. Schmidhuber J (2015) Deep learning in neural networks: an overview[J]. Neural Netw 61:85–117

    Article  Google Scholar 

  27. Shafaey MA, Salem MAM, Al-Berry MN, Ebied HM, Tolba MF (2020) Remote Sensing Image Classification Based on Convolutional Neural Networks. In: Hassanien AE, Azar A, Gaber T, Oliva D, Tolba F (eds) Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020). AICV 2020. Advances in intelligent systems and computing, vol 1153. Springer, Cham

    Google Scholar 

  28. Shahabi H, Shirzadi A, Ghaderi K, Omidvar E, al-Ansari N, Clague JJ, Geertsema M, Khosravi K, Amini A, Bahrami S, Rahmati O, Habibi K, Mohammadi A, Nguyen H, Melesse AM, Ahmad BB, Ahmad A (2020) Flood detection and susceptibility mapping using sentinel-1 remote sensing data and a machine learning approach: hybrid intelligence of bagging ensemble based on k-nearest neighbor classifier[J]. Remote Sens 12(2):266

    Article  Google Scholar 

  29. Sowmya V, Soman KP, Hassaballah M (2019) Hyperspectral image: fundamentals and advances[M]//recent advances in computer vision. Springer, Cham, pp 401–424

    Google Scholar 

  30. Uddin MP, Mamun MA, Hossain MA (2020). PCA-based feature reduction for Hyperspectral remote sensing image classification[J]. IETE Tech Rev: 1–21

  31. Wagner FH, Dalagnol R, Tarabalka Y, Segantine TYF, Thomé R, Hirye MCM (2020) U-net-id, an instance segmentation model for building extraction from satellite images—case study in the Joanópolis City, Brazil[J]. Remote Sens 12(10):1544

    Article  Google Scholar 

  32. Wan Y, Ma A, Zhong Y, Hu X, Zhang L (2020) Multiobjective Hyperspectral feature selection based on discrete sine cosine algorithm[J]. IEEE Trans Geosci Remote Sens 58(5):3601–3618

    Article  Google Scholar 

  33. Wang HH, Tian S W, Yu L, et al. (2020). Bidirectional IndRNN malicious webpages detection algorithm based on convolutional neural network and attention mechanism[J]. Journal of Intelligent & Fuzzy Systems, (Preprint): 1–12

  34. You H, Tian S, Yu L, Lv Y (2019) Pixel-level remote sensing image recognition based on bidirectional word vectors. IEEE Transactions on Geoscience and Remote Sensing, 2020 58(2):1281–1293

    Article  Google Scholar 

  35. Zhang K, Geng X, Yan X H (2020). Prediction of 3-D Ocean temperature by multilayer convolutional LSTM[J]. IEEE Geosci Remote Sens Lett

  36. Zhou W, Ming D, Lv X, Zhou K, Bao H, Hong Z (2020) SO–CNN based urban functional zone fine division with VHR remote sensing image[J]. Remote Sens Environ 236:111458

    Article  Google Scholar 

  37. Zhu X, Bao W (2019) Investigation of remote sensing image fusion strategy applying PCA to wavelet packet analysis based on IHS transform. J Indian Soc Remote Sens 47:413–425

    Article  Google Scholar 

  38. Zhu Z, Geng X, Li S (2020). et al. Ocean surface current retrieval at Hangzhou Bay from Himawari-8 sequential satellite images[J]. Sci China Earth Sci: 1–13

  39. Zhu P, Tan Y, Zhang L, Wang Y, Mei J, Liu H, Wu M (2020) Deep learning for multilabel remote sensing image annotation with dual-level semantic concepts[J]. IEEE Trans Geosci Remote Sens 58(6):4047–4060

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zesong Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Zou, C. & Cui, X. Low-sample size remote sensing image recognition based on a multihead attention integration network. Multimed Tools Appl 79, 32525–32540 (2020). https://doi.org/10.1007/s11042-020-09641-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09641-8

Keywords