Skip to main content
SpringerLink
Log in

Multi-view graph representation learning for hyperspectral image classification with spectral–spatial graph neural networks

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Hyperspectral image (HSI) classification benefits from effectively handling both spectral and spatial features. However, deep learning (DL) models, like graph convolutional networks (GCN), face challenges in computation time, overfitting, and less informative features. To address these challenges, we propose a novel method called MV-GRL (multi-view graph representation learning) for HSI classification using spectral–spatial graph neural networks. MV-GRL incorporates spectral and spatial features with the extended morphological profile (EMP) and employs a multi-view graph autoencoder (MVGAE) to learn a low-dimensional latent graph representation by fusing two input graphs: spectral features-based graph and spatial-based graph. This fusion enhances the discriminative power of the features. Additionally, we introduce a semi-supervised spectral–spatial GCN using the multi-view latent representation, leveraging labeled and unlabeled samples for improved classification performance. By leveraging both labeled and unlabeled data, our method effectively captures underlying relationships and enhances overall accuracy. Experimental results on the Indian Pines, Salinas, and Pavia University datasets demonstrate its competitive performance, achieving overall accuracy (OA) scores of 96.91%, 97.64%, and 98.88%, respectively, surpassing state-of-the-art (SOTA) models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available at [https://ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes].

Notes

  1. https://ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes.

References

  1. Shi Z, Liu L, Zhai X, Jiang Z (2013) Efficient sparse unmixing analysis for hyperspectral imagery based on random projection. Neural Comput Appl 23(7–8):2281–2293. https://doi.org/10.1007/s00521-012-1179-8

    Article  Google Scholar 

  2. Zhao J, Huang T, Zhou Z (2020) Hyperspectral image super-resolution using recursive densely convolutional neural network with spatial constraint strategy. Neural Comput Appl 32(18):14471–14481. https://doi.org/10.1007/s00521-019-04484-3

    Article  Google Scholar 

  3. Krishna SL, Jeya IJS, Deepa SN (2022) Fuzzy-twin proximal SVM kernel-based deep learning neural network model for hyperspectral image classification. Neural Comput Appl 34(21):19343–19376. https://doi.org/10.1007/s00521-022-07517-6

    Article  Google Scholar 

  4. Gao H, Yang Y, Li C, Zhang X, Zhao J, Yao D (2019) Convolutional neural network for spectral-spatial classification of hyperspectral images. Neural Comput Appl 31(12):8997–9012. https://doi.org/10.1007/s00521-019-04371-x

    Article  Google Scholar 

  5. Bandyopadhyay M (2021) Multi-stack hybrid CNN with non-monotonic activation functions for hyperspectral satellite image classification. Neural Comput Appl 33(21):14809–14822. https://doi.org/10.1007/s00521-021-06120-5

    Article  Google Scholar 

  6. Sellami A, Farah M, Dalla Mura M (2023) Shcnet: a semi-supervised hypergraph convolutional networks based on relevant feature selection for hyperspectral image classification. Pattern Recogn Lett 165:98–106

    Article  ADS  Google Scholar 

  7. Firat H, Asker ME, Bayindir MI, Hanbay D (2023) 3d residual spatial-spectral convolution network for hyperspectral remote sensing image classification. Neural Comput Appl 35(6):4479–4497. https://doi.org/10.1007/s00521-022-07933-8

    Article  Google Scholar 

  8. Luo H, Tang YY, Yang X, Yang L, Li H (2017) Autoencoder with extended morphological profile for hyperspectral image classification. In: 2017 3rd IEEE international conference on cybernetics (CYBCONF), pp 1–4. https://doi.org/10.1109/CYBConf.2017.7985761

  9. Kang X, Li C, Li S, Lin H (2018) Classification of hyperspectral images by gabor filtering based deep network. IEEE J Select Top Appl Earth Observ Remote Sens 11(4):1166–1178. https://doi.org/10.1109/JSTARS.2017.2767185

    Article  ADS  Google Scholar 

  10. Akyürek HA, Koçer B (2019) Semi-supervised fuzzy neighborhood preserving analysis for feature extraction in hyperspectral remote sensing images. Neural Comput Appl 31(8):3385–3415. https://doi.org/10.1007/s00521-017-3279-y

    Article  Google Scholar 

  11. Gou J, Xue Y, Ma H, Liu Y, Zhan Y, Ke J (2020) Double graphs-based discriminant projections for dimensionality reduction. Neural Comput Appl 32(23):17533–17550. https://doi.org/10.1007/s00521-020-04924-5

    Article  Google Scholar 

  12. Singh PS, Subbiah K (2022) Enhanced classification of remotely sensed hyperspectral images through efficient band selection using autoencoders and genetic algorithm. Neural Comput Appl 34(24):21539–21550. https://doi.org/10.1007/s00521-021-06121-4

    Article  Google Scholar 

  13. Ye M, Ji C, Chen H, Lei L, Lu H, Qian Y (2020) Residual deep pca-based feature extraction for hyperspectral image classification. Neural Comput Appl 32(18):14287–14300. https://doi.org/10.1007/s00521-019-04503-3

    Article  Google Scholar 

  14. Jia W, Sun M, Lian J, Hou S (2022) Feature dimensionality reduction: a review. Complex Intell Syst 10:1–31

    Google Scholar 

  15. Yu J, Liu H, Zheng X (2020) Two-dimensional joint local and nonlocal discriminant analysis-based 2d image feature extraction for deep learning. Neural Comput Appl 32(10):6009–6024. https://doi.org/10.1007/s00521-019-04085-0

    Article  Google Scholar 

  16. Romero A, Gatta C, Camps-Valls G (2016) Unsupervised deep feature extraction for remote sensing image classification. IEEE Trans Geosci Remote Sens 54(3):1349–1362. https://doi.org/10.1109/TGRS.2015.2478379

    Article  ADS  Google Scholar 

  17. Traboulsi YE, Dornaika F (2018) Flexible semi-supervised embedding based on adaptive loss regression: application to image categorization. Inf Sci 444:1–19. https://doi.org/10.1016/j.ins.2018.02.044

    Article  MathSciNet  Google Scholar 

  18. Zhang L, Shan L, Wang J (2017) Optimal feature selection using distance-based discrete firefly algorithm with mutual information criterion. Neural Comput Appl 28(9):2795–2808. https://doi.org/10.1007/s00521-016-2204-0

    Article  Google Scholar 

  19. Salem OAM, Liu F, Chen Y-PP, Hamed A, Chen X (2022) Fuzzy joint mutual information feature selection based on ideal vector. Expert Syst Appl 193:116453. https://doi.org/10.1016/j.eswa.2021.116453

    Article  Google Scholar 

  20. Feng J, Liu L, Cao X, Jiao L, Sun T, Zhang X (2018) Marginal stacked autoencoder with adaptively-spatial regularization for hyperspectral image classification. IEEE J Select Top Appl Earth Observ Remote Sen 11(9):3297–3311. https://doi.org/10.1109/JSTARS.2018.2854893

    Article  ADS  Google Scholar 

  21. Kordi Ghasrodashti E, Sharma N (2021) Hyperspectral image classification using an extended auto-encoder method. Signal Process Image Commun 92:116111. https://doi.org/10.1016/j.image.2020.116111

    Article  Google Scholar 

  22. Shi C, Pun C-M (2017) 3d multi-resolution wavelet convolutional neural networks for hyperspectral image classification. Inf Sci 420:49–65. https://doi.org/10.1016/j.ins.2017.08.051

    Article  Google Scholar 

  23. Kipf TN, Welling M (2016) Variational graph auto-encoders

  24. Hong D, Gao L, Yao J, Zhang B, Plaza A, Chanussot J (2021) Graph convolutional networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59(7):5966–5978. https://doi.org/10.1109/TGRS.2020.3015157

    Article  ADS  Google Scholar 

  25. Ma W, Ma H, Zhu H, Li Y, Li L, Jiao L, Hou B (2021) Hyperspectral image classification based on spatial and spectral kernels generation network. Inf Sci 578:435–456. https://doi.org/10.1016/j.ins.2021.07.043

    Article  MathSciNet  Google Scholar 

  26. Tao C, Pan H, Li Y, Zou Z (2015) Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification. IEEE Geosci Remote Sens Lett 12(12):2438–2442. https://doi.org/10.1109/LGRS.2015.2482520

    Article  ADS  Google Scholar 

  27. Paul S, Nagesh Kumar D (2018) Spectral-spatial classification of hyperspectral data with mutual information based segmented stacked autoencoder approach. ISPRS J Photogrammet Remote Sens 138:265–280

    Article  ADS  Google Scholar 

  28. Chen Y, Zhao X, Jia X (2015) Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE J Select Top Appl Earth Observ Remote Sens 8(6):2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577

    Article  ADS  Google Scholar 

  29. Zhou X, Li S, Tang F, Qin K, Hu S, Liu S (2017) Deep learning with grouped features for spatial spectral classification of hyperspectral images. IEEE Geosci Remote Sens Lett 14(1):97–101. https://doi.org/10.1109/LGRS.2016.2630045

    Article  ADS  Google Scholar 

  30. Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: 2015 IEEE international geoscience and remote sensing symposium (IGARSS), pp 4959–4962. https://doi.org/10.1109/IGARSS.2015.7326945

  31. Cao F, Guo W (2020) Cascaded dual-scale crossover network for hyperspectral image classification. Knowl Based Syst 189:105122. https://doi.org/10.1016/j.knosys.2019.105122

    Article  Google Scholar 

  32. Sellami A, Farah M, Riadh Farah I, Solaiman B (2019) Hyperspectral imagery classification based on semi-supervised 3-d deep neural network and adaptive band selection. Expert Syst Appl 129:246–259. https://doi.org/10.1016/j.eswa.2019.04.006

    Article  Google Scholar 

  33. Sellami A, Ben Abbes A, Barra V, Farah IR (2020) Fused 3-d spectral-spatial deep neural networks and spectral clustering for hyperspectral image classification. Pattern Recogn Lett 138:594–600. https://doi.org/10.1016/j.patrec.2020.08.020

    Article  ADS  Google Scholar 

  34. Qin A, Shang Z, Tian J, Wang Y, Zhang T, Tang YY (2019) Spectral-spatial graph convolutional networks for semisupervised hyperspectral image classification. IEEE Geosci Remote Sens Lett 16(2):241–245. https://doi.org/10.1109/LGRS.2018.2869563

    Article  ADS  Google Scholar 

  35. Wan S, Gong C, Zhong P, Pan S, Li G, Yang J (2020) Hyperspectral image classification with context-aware dynamic graph convolutional network. IEEE Trans Geosci Remote Sens 59(1):597–612. https://doi.org/10.1109/TGRS.2020.2994205

    Article  ADS  Google Scholar 

  36. Wan S, Gong C, Zhong P, Du B, Zhang L, Yang J (2020) Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58(5):3162–3177. https://doi.org/10.1109/TGRS.2019.2949180

    Article  ADS  Google Scholar 

  37. Mou L, Lu X, Li X, Zhu XX (2020) Nonlocal graph convolutional networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58(12):8246–8257

    Article  ADS  Google Scholar 

  38. Wang H, Cheng Y, Chen CP, Wang X (2021) Semisupervised classification of hyperspectral image based on graph convolutional broad network. IEEE J Select Top Appl Earth Observ Remote Sens 14:2995–3005

    Article  ADS  Google Scholar 

  39. Wan S, Pan S, Zhong P, Chang X, Yang J, Gong C (2021) Dual interactive graph convolutional networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–14. https://doi.org/10.1109/TGRS.2021.3075223

    Article  Google Scholar 

  40. Bai J, Ding B, Xiao Z, Jiao L, Chen H, Regan AC (2021) Hyperspectral image classification based on deep attention graph convolutional network. IEEE Trans Geosci Remote Sens 60:1–16. https://doi.org/10.1109/TGRS.2021.3066485

    Article  Google Scholar 

  41. Zuo X, Yu X, Liu B, Zhang P, Tan X (2022) Fsl-egnn: edge-labeling graph neural network for hyperspectral image few-shot classification. IEEE Trans Geosci Remote Sens 60:1–18. https://doi.org/10.1109/TGRS.2022.3165025

    Article  Google Scholar 

  42. He X, Chen Y, Ghamisi P (2022) Dual graph convolutional network for hyperspectral image classification with limited training samples. IEEE Trans Geosci Remote Sens 60:1–18. https://doi.org/10.1109/TGRS.2021.3061088

    Article  Google Scholar 

  43. Liu W, Liu B, He P, Hu Q, Gao K, Li H (2023) Masked graph convolutional network for small sample classification of hyperspectral images. Remote Sens. https://doi.org/10.3390/rs15071869

    Article  Google Scholar 

  44. Bhatti UA, Huang M, Neira-Molina H, Marjan S, Baryalai M, Tang H, Wu G, Bazai SU (2023) Mffcg-multi feature fusion for hyperspectral image classification using graph attention network. Expert Syst Appl 229:120496. https://doi.org/10.1016/j.eswa.2023.120496

    Article  Google Scholar 

  45. Aptoula E, Ozdemir MC, Yanikoglu B (2016) Deep learning with attribute profiles for hyperspectral image classification. IEEE Geosci Remote Sens Lett 13(12):1970–1974. https://doi.org/10.1109/LGRS.2016.2619354

    Article  ADS  Google Scholar 

  46. Dalla Mura M, Villa A, Benediktsson JA, Chanussot J, Bruzzone L (2011) Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis. IEEE Geosci Remote Sens Lett 8(3):542–546. https://doi.org/10.1109/LGRS.2010.2091253

    Article  ADS  Google Scholar 

  47. Dalla Mura M, Benediktsson JA, Waske B, Bruzzone L (2010) Morphological attribute profiles for the analysis of very high resolution images. IEEE Geosci Remote Sens Lett 48(10):3747–3762. https://doi.org/10.1109/TGRS.2010.2048116

    Article  ADS  Google Scholar 

Download references

Funding

Funding information is not applicable/no funding was received.

Author information

Author notes

  1. Akrem Sellami, Imed Riadh Farah and Mauro Dalla Mura contributed equally to this work.

Authors and Affiliations

  1. RIADI Laboratory, ENSI, University of Manouba, 2010, Manouba, Tunis, Tunisia

    Refka Hanachi & Imed Riadh Farah

  2. Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, 59000, Lille, France

    Akrem Sellami

  3. ITI Department, IMT Atlantique, 655 Avenue du Technopôle, 29280, Plouzané, Paris, France

    Imed Riadh Farah

  4. Univ. Grenoble Alpes, CNRS, Grenoble INP (Institute of Engineering Univ. Grenoble Alpes), GIPSA-lab, 38000, Grenoble, France

    Mauro Dalla Mura

  5. Institut Universitaire de France (IUF), 75231, Paris, France

    Mauro Dalla Mura

Authors
  1. Refka Hanachi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Akrem Sellami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Imed Riadh Farah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mauro Dalla Mura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Refka Hanachi.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hanachi, R., Sellami, A., Farah, I.R. et al. Multi-view graph representation learning for hyperspectral image classification with spectral–spatial graph neural networks. Neural Comput & Applic 36, 3737–3759 (2024). https://doi.org/10.1007/s00521-023-09275-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09275-5

Keywords

Search

Navigation