Abstract
For a long time, small sample recognition for hyperspectral images has been a popular research topic. It is very difficult for an algorithm to simultaneously satisfy the requirements of feature mining, feature selection and feature integration. The traditional single model has difficulty completing multiple tasks at the same time, ultimately leading to poor small sample recognition results for remote sensing images. This paper proposes a multimodel joint algorithm for deep feature mining based on multiscale convolution (MC) under multihead attention (MA) and deep feature integration based on bidirectional independent recurrent neural networks (BiIndRNNs), MACBINet. First, this paper proposes a multihead attention mechanism that assigns multiple weight coefficients to each feature to better select remote sensing image features; then, it implements the deep mining of features and the retention of multiple deep features through multiscale convolution. Subsequently, it implements contextual semantic information integration for long-sequence features through bidirectional independent recurrent neural networks to avoid the problem of gradient disappearance during training on a small sample of data. Finally, the softmax function is used to perform recognition on three public remote sensing data sets. The experimental results prove that our proposed MACBINet achieves the best results to date for small sample classification.
Similar content being viewed by others
References
Achour S, Chikr Elmezouar M, Taleb N, Kpalma K., Ronsin J. (2020). A PCA-PD fusion method for change detection in remote sensing multi temporal images[J]. Geocarto Int : 1–18
Akhlaq MLM, Winarso G (2020). Comparative analysis of object-based and pixel-based classification of high-resolution remote sensing images for mapping coral reef geomorphic zones[C]//1st Borobudur international symposium on humanities, economics and social sciences (BIS-HESS 2019). Atlantis Press: 992–996
Anders K, Winiwarter L, Lindenbergh R, Williams JG, Vos SE, Höfle B (2020) 4D objects-by-change: spatiotemporal segmentation of geomorphic surface change from LiDAR time series[J]. ISPRS J Photogramm Remote Sens 159:352–363
Bakhti K, Arabi MEA, Chaib S et al (2020) Bi-directional LSTM model for classification of vegetation from satellite time series[C]//2020 Mediterranean and middle-east geoscience and remote sensing symposium (M2GARSS). IEEE:160–163
Bioucas-Dias JM, Plaza A, Camps-Valls G, Scheunders P, Nasrabadi N, Chanussot J (2013) Hyperspectral remote sensing data analysis and future challenges[J]. IEEE Geoscience and remote sensing magazine 1(2):6–36
Cai W, Wei Z (2020) PiiGAN: generative adversarial networks for pluralistic image Inpainting. IEEE Access 8:48451–48463
Chen J, Chen S, Zhou P, Qian Y (2019) Deep neural network based Hyperspectral pixel classification with factorized spectral-spatial feature representation. IEEE Access 7:81407–81418
Chen C, He X, Chu Y et al (2020) A new remote sensing image fusion method combining principal component analysis and curvelet transform[J]. MS&E 780(3):032054
Chen T, Zhao Y, Guo Y (2020) Sparsity-regularized feature selection for multi-class remote sensing image classification. Neural Comput & Applic 32:6513–6521
Cui J, Zhang X, Wang W, Wang L (2020) Integration of optical and SAR remote sensing images for crop-type mapping based on a novel object-oriented feature selection method[J]. International Journal of Agricultural and Biological Engineering 13(1):178–190
Deep Learning in Computer Vision: Principles and Applications[M]. CRC Press, 2020.
Feng J, Wu X, Chen J, et al. (2019). “Joint multilayer spatial-spectral Classi-fication of Hyperspectral images based on CNN and Convlstm,” IEEE International Geoscience and Remote Sensing Symposium IEEE: 588–591
Gao H, Yang Y, Yao D, Li C (2019) Hyperspectral image classification with pre-activation residual attention network[J]. IEEE Access 7:176587–176599
Ghaffari R, Golpardaz M, Helfroush MS, Danyali H (2020) A fast, weighted CRF algorithm based on a two-step superpixel generation for SAR image segmentation[J]. Int J Remote Sens 41(9):3535–3557
Huang M, Chen Q, Wang H (2020) A multivariable optical remote sensing image feature discretization method applied to marine vessel targets recognition[J]. Multimed Tools Appl 79(7):4597–4618
Ji C, Ye M, Lu H, et al. (2019). Feature Extraction of Hyperspectral Imagery Based on Deep NMF[C]//IGARSS 2019-2019 IEEE international geoscience and remote sensing symposium. IEEE: 1092–1095
LeCun Y, Bengio Y, Hinton G (2015) Deep learning[J]. Nature 521(7553):436–444
Lee H, Kwon H (2017) Going deeper with contextual CNN for hyperspectral image classification. IEEE Transactions on Image Processing 26(10):4843–4855
Lei, L. X., & Peng, L. (2020). Training strategy of CNN for remote sensing image classification with active learning. May, 52(1)
Li J, Bioucas-Dias JM, Plaza A (2012) Semisupervised hyperspectral image classification using soft sparse multinomial logistic regression[J]. IEEE Geosci Remote Sens Lett 10(2):318–322
Li Y, Fang S, Jiao L, Liu R, Shang R (2020) A multi-level attention model for remote sensing image captions[J]. Remote Sens 12(6):939
Li P, Han L, Tao X, et al. (2020). Hashing nets for hashing: a quantized deep learning to hash framework for remote sensing image retrieval[J]. IEEE Trans Geosci Remote Sens
Li G, Zhang C, Lei R, Zhang X, Ye Z, Li X (2020) Hyperspectral remote sensing image classification using three-dimensional-squeeze-and-excitation-DenseNet (3D-SE-DenseNet)[J]. Remote Sensing Letters 11(2):195–203
Liu X, Zhou Y, Zhao J, Yao R, Liu B, Ma D, Zheng Y (2020) Multiobjective ResNet pruning by means of EMOAs for remote sensing scene classification[J]. Neurocomputing 381:298–305
Pan E, Mei X, Wang Q, Ma Y, Ma J (2020) Spectral-spatial classification for hyperspectral image based on a single GRU[J]. Neurocomputing 387:150–160
Schmidhuber J (2015) Deep learning in neural networks: an overview[J]. Neural Netw 61:85–117
Shafaey MA, Salem MAM, Al-Berry MN, Ebied HM, Tolba MF (2020) Remote Sensing Image Classification Based on Convolutional Neural Networks. In: Hassanien AE, Azar A, Gaber T, Oliva D, Tolba F (eds) Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020). AICV 2020. Advances in intelligent systems and computing, vol 1153. Springer, Cham
Shahabi H, Shirzadi A, Ghaderi K, Omidvar E, al-Ansari N, Clague JJ, Geertsema M, Khosravi K, Amini A, Bahrami S, Rahmati O, Habibi K, Mohammadi A, Nguyen H, Melesse AM, Ahmad BB, Ahmad A (2020) Flood detection and susceptibility mapping using sentinel-1 remote sensing data and a machine learning approach: hybrid intelligence of bagging ensemble based on k-nearest neighbor classifier[J]. Remote Sens 12(2):266
Sowmya V, Soman KP, Hassaballah M (2019) Hyperspectral image: fundamentals and advances[M]//recent advances in computer vision. Springer, Cham, pp 401–424
Uddin MP, Mamun MA, Hossain MA (2020). PCA-based feature reduction for Hyperspectral remote sensing image classification[J]. IETE Tech Rev: 1–21
Wagner FH, Dalagnol R, Tarabalka Y, Segantine TYF, Thomé R, Hirye MCM (2020) U-net-id, an instance segmentation model for building extraction from satellite images—case study in the Joanópolis City, Brazil[J]. Remote Sens 12(10):1544
Wan Y, Ma A, Zhong Y, Hu X, Zhang L (2020) Multiobjective Hyperspectral feature selection based on discrete sine cosine algorithm[J]. IEEE Trans Geosci Remote Sens 58(5):3601–3618
Wang HH, Tian S W, Yu L, et al. (2020). Bidirectional IndRNN malicious webpages detection algorithm based on convolutional neural network and attention mechanism[J]. Journal of Intelligent & Fuzzy Systems, (Preprint): 1–12
You H, Tian S, Yu L, Lv Y (2019) Pixel-level remote sensing image recognition based on bidirectional word vectors. IEEE Transactions on Geoscience and Remote Sensing, 2020 58(2):1281–1293
Zhang K, Geng X, Yan X H (2020). Prediction of 3-D Ocean temperature by multilayer convolutional LSTM[J]. IEEE Geosci Remote Sens Lett
Zhou W, Ming D, Lv X, Zhou K, Bao H, Hong Z (2020) SO–CNN based urban functional zone fine division with VHR remote sensing image[J]. Remote Sens Environ 236:111458
Zhu X, Bao W (2019) Investigation of remote sensing image fusion strategy applying PCA to wavelet packet analysis based on IHS transform. J Indian Soc Remote Sens 47:413–425
Zhu Z, Geng X, Li S (2020). et al. Ocean surface current retrieval at Hangzhou Bay from Himawari-8 sequential satellite images[J]. Sci China Earth Sci: 1–13
Zhu P, Tan Y, Zhang L, Wang Y, Mei J, Liu H, Wu M (2020) Deep learning for multilabel remote sensing image annotation with dual-level semantic concepts[J]. IEEE Trans Geosci Remote Sens 58(6):4047–4060
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, Z., Zou, C. & Cui, X. Low-sample size remote sensing image recognition based on a multihead attention integration network. Multimed Tools Appl 79, 32525–32540 (2020). https://doi.org/10.1007/s11042-020-09641-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09641-8