Skip to main content
SpringerLink
Log in

Hyperspectral image classification using multi-level features fusion capsule network with a dense structure

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The convolution neural network (CNN) methods have achieved excellent performance in hyperspectral image (HSI) classification. However, the convolution network fails to utilize the relative position information of the image effectively. The emergence of the capsule network has solved this limitation and made significant progress in the field of HSI classification. However, most capsule-based methods stack convolution layers to extract feature information, which can only get in-depth information while losing shallow information. In this paper, we proposed a multi-level features fusion capsule network based on dense structure (MLFF-CapsNet). In this framework, we designed a dense block composed of four-layer convolutions to fully extract spectral and spatial features. Then, each extracted feature map is concatenated to form multi-level features fusion capsules, which are transported to the dynamic routing algorithm to obtain the prediction category. The model presented has strong characterization and generalization capabilities under a few labeled samples. Experimental results on three hyperspectral datasets demonstrate that the proposed method achieves superior classification performance over the advanced comparison models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Zhang X, Sun Y, Shang K, Zhang L, Wang S (2016) Crop classification based on feature band set construction and object-oriented approach using hyperspectral images. IEEE J Sel Top Appl Earth Obs Remote Sens 9(9):4117–4128

    Article  Google Scholar 

  2. Manjunath KR, Ray SS, Vyas D (2016) Identification of indices for accurate estimation of anthocyanin and carotenoids in different species of flowers using hyperspectral data. Remote Sens Lett

  3. Yu L, Hong Y, Zhou Y, Zhu Q, Nie Y (2016) Wavelength variable selection methods for estimation of soil organic matter content using hyperspectral technique. Trans Chinese Soc Agricultur Eng

  4. Uzkent B, Rangnekar A, Hoffman MJ (2017) Aerial vehicle tracking by adaptive fusion of hyperspectral likelihood maps. IEEE

  5. Liu M, Zhao J, Li G, Zhang H, Wu T (2017) Tongue coat information extraction of the traditional chinese medicine with hyperspectral image. Guang pu xue yu Guang pu fen xi= Guang pu 37(1):162–165

    Google Scholar 

  6. Nalepa J, Myller M, Imai Y, Honda K-I, Takeda T, Antoniak M (2020) Unsupervised segmentation of hyperspectral images using 3-d convolutional autoencoders. IEEE Geosci Remote Sens Lett 17(11):1948–1952

    Article  Google Scholar 

  7. Zhang L, Zhang L, Du B (2016) Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sens Magazine 4(2):22–40

    Article  Google Scholar 

  8. Zhang L, Zhang L, Tao D, Huang X, Du B (2013) Hyperspectral remote sensing image subpixel target detection based on supervised metric learning. IEEE Trans Geoscience Remote Sens 52(8):4955–4965

    Article  Google Scholar 

  9. Licciardi G, Marpu PR, Chanussot J, Benediktsson JA (2011) Linear versus nonlinear pca for the classification of hyperspectral data based on the extended morphological profiles. IEEE Geosci Remote Sens Lett 9(3):447–451

    Article  Google Scholar 

  10. Deng Y-J, Li H-C, Pan L, Shao L-Y, Du Q, Emery WJ (2018) Modified tensor locality preserving projection for dimensionality reduction of hyperspectral images. IEEE Geosci Remote Sens Lett 15 (2):277–281

    Article  Google Scholar 

  11. Villa A, Benediktsson JA, Chanussot J, Jutten C (2011) Hyperspectral image classification with independent component discriminant analysis. IEEE Trans Geosci Remote Sens 49(12):4865–4876

    Article  Google Scholar 

  12. Liu J, Wu Z, Wei Z, Xiao L, Sun L (2013) Spatial-spectral kernel sparse representation for hyperspectral image classification. IEEE J Sel Top Appl Earth Obs Remote Sens 6(6):2462–2471

    Article  Google Scholar 

  13. Gao L, Hong D, Yao J, Zhang B, Gamba P, Chanussot J (2020) Spectral superresolution of multispectral imagery with joint sparse and low-rank learning. IEEE Trans Geosci Remote Sens

  14. Fauvel M, Benediktsson JA, Chanussot J, Sveinsson JR (2008) Spectral and spatial classification of hyperspectral data using svms and morphological profiles. IEEE Trans Geosci Remote Sens 46 (11):3804–3814

    Article  Google Scholar 

  15. Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans Geosci Remote Sens 42(8):1778–1790

    Article  Google Scholar 

  16. Zhao Y, Qian Y, Li C (2017) Improved knn text classification algorithm with mapreduce implementation. In: 2017 4th international conference on systems and informatics (ICSAI). IEEE, pp 1417-1422

  17. Li J, Bioucas-Dias JM, Plaza A (2011) Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and markov random fields. IEEE Trans Geosci Remote Sens 50(3):809–823

    Article  Google Scholar 

  18. Li W, Chen C, Su H, Du Q (2015) Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Trans Geosci Remote Sens 53(7):3681–3693

    Article  Google Scholar 

  19. Deng S, Xu Y, He Y, Yin J, Wu Z (2015) A hyperspectral image classification framework and its application. Inform Sci 299:379–393

    Article  Google Scholar 

  20. Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54 (10):6232–6251

    Article  Google Scholar 

  21. Diao W, Sun X, Zheng X, Dou F, Wang H, Fu K (2016) Efficient saliency-based object detection in remote sensing images using deep belief networks. IEEE Geosci Remote Sens Lett 13(2):137–141

    Article  Google Scholar 

  22. Rao M, Tang P, Zhang Z (2020) A developed siamese cnn with 3d adaptive spatial-spectral pyramid pooling for hyperspectral image classification. Remote Sens 12(12):1964

    Article  Google Scholar 

  23. Wu P, Cui Z, Gan Z, Liu F (2020) Residual group channel and space attention network for hyperspectral image classification. Remote Sens 12(12):2035

    Article  Google Scholar 

  24. Mou L, Zhu XX (2019) Learning to pay attention on spectral domain: a spectral attention module-based convolutional network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58 (1):110–122

    Article  Google Scholar 

  25. Wang L, Peng J, Sun W (2019) Spatial–spectral squeeze-and-excitation residual network for hyperspectral image classification. Remote Sens 11(7):884

    Article  Google Scholar 

  26. Zhong Z, Li J, Clausi DA, Wong A (2019) Generative adversarial networks and conditional random fields for hyperspectral image classification. IEEE Trans Cybern 50(7):3318–3329

    Article  Google Scholar 

  27. Yuan Y, Zheng X, Lu X (2017) Hyperspectral image superresolution by transfer learning. IEEE J Select Topics Appl Earth Observ Remote Sens 10(5):1963–1974

    Article  Google Scholar 

  28. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) Hybridsn: exploring 3-d–2-d cnn feature hierarchy for hyperspectral image classification. IEEE Geosci Remote Sens Lett 17(2):277–281

    Article  Google Scholar 

  29. Fang L, Li S, Kang X, Benediktsson JA (2015) Spectral–spatial classification of hyperspectral images with a superpixel-based discriminative sparse model. IEEE Trans Geosci Remote Sens 53(8):4186–4201

    Article  Google Scholar 

  30. Li R, Zheng S, Duan C, Yang Y, Wang X (2020) Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sens 12(3):582

    Article  Google Scholar 

  31. Zhong Z, Li J, Luo Z, Chapman M (2017) Spectral–spatial residual network for hyperspectral image classification: a 3-d deep learning framework. IEEE Trans Geosci Remote Sens 56(2):847–858

    Article  Google Scholar 

  32. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. arXiv:1710.09829

  33. Kosiorek AR, Sabour S, Teh YW, Hinton GE (2019) Stacked capsule autoencoders. arXiv:1906.06818

  34. Liu J-w, Gao F, Lu R-k, Lian Y-f, Wang D-z, Luo X-l, Wang C-R (2019) Ddrm-capsnet: capsule network based on deep dynamic routing mechanism for complex data. In: International conference on artificial neural networks. Springer, pp 178–189

  35. Wang W-Y, Li H-C, Pan L, Yang G, Du Q (2018) Hyperspectral image classification based on capsule network. In: IGARSS 2018-2018 IEEE international geoscience and remote sensing symposium. IEEE, pp 3571–3574

  36. Tian T, Liu X, Wang L (2019) Remote sensing scene classification based on res-capsnet. In: IGARSS 2019-2019 IEEE international geoscience and remote sensing symposium. IEEE, pp 525-528

  37. Wang X, Tan K, Chen Y (2018) Capsnet and triple-gans towards hyperspectral classification. In: 2018 Fifth International Workshop on Earth Observation and Remote Sensing Applications (EORSA). IEEE, pp 1–4

  38. Xue Z (2020) A general generative adversarial capsule network for hyperspectral image spectral-spatial classification. Remote Sens Lett 11(1):19–28

    Article  Google Scholar 

  39. Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z (2015) Deeply-supervised nets. In: Artificial intelligence and statistics. PMLR, pp 562–570

  40. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  41. Zhang H, Meng L, Wei X, Tang X, Tang X, Wang X, Jin B, Yao W (2019) 1d-convolutional capsule network for hyperspectral image classification. arXiv:1903.09834

  42. Khodadadzadeh M, Ding X, Chaurasia P, Coyle D (2021) A hybrid capsule network for hyperspectral image classification. IEEE J Sel Top Appl Earth Obs Remote Sens 14:11824–11839

    Article  Google Scholar 

  43. Zhu K, Chen Y, Ghamisi P, Jia X, Benediktsson JA (2019) Deep convolutional capsule network for hyperspectral image spectral and spectral-spatial classification. Remote Sens 11(3):223

    Article  Google Scholar 

  44. Lei R, Zhang C, Liu W, Zhang L, Zhang X, Yang Y, Huang J, Li Z, Zhou Z (2021) Hyperspectral remote sensing image classification using deep convolutional capsule network. IEEE J Sel Top Appl Earth Observ Remote Sens 14:8297–8315

    Article  Google Scholar 

  45. Wang X, Ren J, Wang R, Wu W, Chen J (2021) Spatial–spectral hyperspectral image classification based on primary and secondary capsule network. J Appl Remote Sens 15(3):036518

    Article  Google Scholar 

  46. Lei R, Zhang C, Du S, Wang C, Zhang X, Zheng H, Huang J, Yu M (2021) A non-local capsule neural network for hyperspectral remote sensing image classification. Remote Sens Lett 12(1):40–49

    Article  Google Scholar 

  47. Lei R, Zhang C, Zhang X, Huang J, Li Z, Liu W, Cui H (2022) Multiscale feature aggregation capsule neural network for hyperspectral remote sensing image classification. Remote Sens 14(7):1652

    Article  Google Scholar 

  48. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

  49. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25:1097–1105

    Google Scholar 

  50. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  51. Story M, Congalton RG (1986) Accuracy assessment: a user’s perspective. Photogramm Eng Remote Sens 52(3):397–399

    Google Scholar 

  52. Kingma DP, Ba J (2014) A method for stochastic optimization, arXiv:1412.6980

  53. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch

  54. Zhang C, Li G, Du S, Tan W, Gao F (2019) Three-dimensional densely connected convolutional network for hyperspectral remote sensing image classification. J Appl Remote Sens 13(1):016519

    Article  Google Scholar 

  55. Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2020) Hybridsn: exploring 3d-2d cnn feature hierarchy for hyperspectral image classification. IEEE Geosci Remote Sens Lett 17(2):277–281

    Article  Google Scholar 

Download references

Acknowledgment

This work described in this paper was supported by the Open Fund of Hubei Key Laboratory of Intelligent Geo-Information Processing (ZRIGIP-201801).

Author information

Authors and Affiliations

  1. School of Computer Science, China University of Geosciences, Wuhan, 430078, China

    Jiansi Ren, Meilin Shi, Jiannan Chen, Ruoxiang Wang & Xin Wang

  2. Hubei Key Laboratory of Intelligent Geo-Information Processing, China University of Geosciences, Wuhan, 430078, China

    Jiansi Ren

Authors
  1. Jiansi Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meilin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiannan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruoxiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiansi Ren.

Ethics declarations

Conflict of Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, J., Shi, M., Chen, J. et al. Hyperspectral image classification using multi-level features fusion capsule network with a dense structure. Appl Intell 53, 14162–14181 (2023). https://doi.org/10.1007/s10489-022-04232-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04232-6

Keywords

Search

Navigation