Abstract
With the widespread and successful application of hyperspectral imaging, the task of classifying hyperspectral images has become a meaningful endeavor. This paper introduces a new approach to clustering hyperspectral images called CTNSC, which incorporates an attention mechanism to improve performance. In comparison to many current convolution-based methods for deep subspace clustering, our approach incorporates an attention mechanism to effectively capture the extensive spatial relationships that exist between objects in hyperspectral images. We combine convolution and attention through the use of Conformer blocks, enabling the network to simultaneously capture both local and global features of objects. As a result, our network can identify a more optimal deep affinity matrix, which can be utilized for spectral clustering to achieve better clustering results. We conducted experiments on three real-world datasets, and the results showed that CTNSC achieved excellent clustering performance compared to many commonly used clustering methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The datasets involved in this paper are all public datasets
Change history
03 August 2023
A Correction to this paper has been published: https://doi.org/10.1007/s12145-023-01064-x
References
Audebert N, Le Saux B, Lefèvre S (2019) Deep learning for classification of hyperspectral data: A comparative review. IEEE Geosci Remote Sens Mag 7(2):159–173
Signoroni, A, Savardi, M, Baronio, A, Benini, S (2019) Deep learning meets hyperspectral image analysis: A multidisciplinary review. J Imaging 5(5) https://doi.org/10.3390/jimaging5050052
Vali, A, Comai, S, Matteucci, M (2020) Deep learning for land use and land cover classification based on hyperspectral and multispectral earth observation data: A review. Remote Sens 12(15) https://doi.org/10.3390/rs12152495
Patro RN, Subudhi S, Biswal PK, Dell’acqua F (2021) A review of unsupervised band selection techniques: Land cover classification for hyperspectral earth observation data. IEEE Geosci Remote Sens Mag 9(3):72–111. https://doi.org/10.1109/MGRS.2021.3051979
Pascucci, S, Pignatti, S, Casa, R, Darvishzadeh, R, Huang, W (2020) Special issue "hyperspectral remote sensing of agriculture and vegetation". Remote Sens 12(21) https://doi.org/10.3390/rs12213665
Hennessy, A, Clarke, K, Lewis, M (2020) Hyperspectral classification of plants: A review of waveband selection generalisability. Remote Sens 12(1) https://doi.org/10.3390/rs12010113
Peyghambari S, Zhang Y (2021) Hyperspectral remote sensing in lithological mapping, mineral exploration, and environmental geology: an updated review. J Appl Remote Sens 15(3):031501–031501
Krupnik D (2019) Khan, S (2019) Close-range, ground-based hyperspectral imaging for mining applications at various scales: Review and case studies. Earth Sci Rev 198:102952
Stuart MB, McGonigle AJ, Willmott JR (2019) Hyperspectral imaging in environmental monitoring: A review of recent developments and technological advances in compact field deployable systems. Sensors 19(14):3071
Kuras A, Brell M, Rizzi J, Burud I (2021) Hyperspectral and lidar data applied to the urban land cover machine learning and neural-network-based classification: A review. Remote Sens 13(17):3393
Lv, W, Wang, X (2020) Overview of hyperspectral image classification. J Sensors 2020
Paoletti M, Haut J, Plaza J, Plaza A (2019) Deep learning classifiers for hyperspectral imaging: A review. ISPRS J Photogramm Remote Sens 158:279–317
Ozdemir A, Polat K (2020) Deep learning applications for hyperspectral imaging: a systematic review. J Inst Electr Comput 2(1):39–56
Huang B, Wang Z, Shang J, Chen G, Radenkovic M (2022) A spectral sequencebased nonlocal long short-term memory network for hyperspectral image classification. IEEE J Sel Top Appl Earth Obs Remote Sens 15:3041–3051. https://doi.org/10.1109/JSTARS.2022.3159729
Manian V, Alfaro-Mejía E, Tokars RP (2022) Hyperspectral image labeling and classification using an ensemble semi-supervised machine learning approach. Sensors 22(4):1623
Yao, D, Zhi-li, Z, Xiao-feng, Z, Wei, C, Fang, H, Yao-ming, C, Cai, W-W (2022) Deep hybrid: multi-graph neural network collaboration for hyperspectral image classification. Defence Technology
Sun H, Zheng X, Lu X (2021) A supervised segmentation network for hyperspectral image classification. IEEE Trans Image Process 30:2810–2825
Sellami A, Tabbone S (2022) Deep neural networks-based relevant latent representation learning for hyperspectral image classification. Pattern Recognit 121:108224
Yue J, Fang L, Rahmani H, Ghamisi P (2021) Self-supervised learning with adaptive distillation for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–13
Wang Q, Chen M, Zhang J, Kang S, Wang Y (2022) Improved active deep learning for semi-supervised classification of hyperspectral image. Remote Sens 14(1):171
Wei W, Xu S, Zhang L, Zhang J, Zhang Y (2021) Boosting hyperspectral image classification with unsupervised feature learning. IEEE Trans Geosci Remote Sens 60:1–15
Elhamifar E, Vidal R (2013) Sparse subspace clustering: Algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781. https://doi.org/10.1109/TPAMI.2013.57
Chen J, Yang J (2014) Robust subspace segmentation via low-rank representation. IEEE Trans Cybern 44(8):1432–1445. https://doi.org/10.1109/TCYB.2013.2286106
Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184. https://doi.org/10.1109/TPAMI.2012.88
Vidal R, Favaro P (2014) Low rank subspace clustering (lrsc). Pattern Recognit Lett 43:47–61. https://doi.org/10.1016/j.patrec.2013.08.006. ICPR2012 Awarded Papers
Lu, C.-Y, Min, H, Zhao, Z-Q, Zhu, L, Huang, D-S, Yan, S (2012) Robust and efficient subspace segmentation via least squares regression. In: Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy,October 7-13, 2012, Proceedings, Part VII 12, 347–360. Springer
Ji, P, Salzmann, M, Li, H (2014) Efficient dense subspace clustering. In: IEEE Winter Conference on Applications of Computer Vision, 461–468. IEEE
Li C-G, You C, Vidal R (2017) Structured sparse subspace clustering: A joint affinity learning and subspace clustering framework. IEEE Trans Image Process 26(6):2988–3001. https://doi.org/10.1109/TIP.2017.2691557
Liu, S, Wang, H (2022) Geometric weighting subspace clustering on nonlinear manifolds. Multimed. Tools. App 81:42971–42990
Zhang H, Zhai H, Zhang L, Li P (2016) Spectral-spatial sparse subspace clustering for hyperspectral remote sensing images. IEEE Trans Geosci Remote Sens 54(6):3672–3684. https://doi.org/10.1109/TGRS.2016.2524557
Huang, S, Zhang, H, Pižurica, A (2018) Joint sparsity based sparse subspace clustering for hyperspectral images. In: 2018 25th IEEE International Conference on Image Processing (ICIP), 3878–3882 https://doi.org/10.1109/ICIP.2018.8451277
Wang L, Niu S, Gao X, Liu K, Lu F, Diao Q, Dong J (2021) Fast high-order sparse subspace clustering with cumulative mrf for hyperspectral images. IEEE Geosci Remote Sens Lett 18(1):152–156. https://doi.org/10.1109/LGRS.2020.2968350
Bruna, J, Zaremba, W, Szlam, A, LeCun, Y (2013) Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203
Defferrard, M, Bresson, X, Vandergheynst, P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv neural inf process sys 29
Kipf, T.N, Welling, M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907
Cai Y, Zhang Z, Cai Z, Liu X, Jiang X, Yan Q (2021) Graph convolutional subspace clustering: A robust subspace clustering framework for hyperspectral image. IEEE Trans Geosci Remote Sens 59(5):4191–4202. https://doi.org/10.1109/TGRS.2020.3018135
Liu S, Wang H (2022) Graph convolutional optimal transport for hyperspectral image spectral clustering. IEEE Trans Geosci Remote Sens 60:1–13. https://doi.org/10.1109/TGRS.2022.3203481
Bacca, J, Hinojosa, C.A, Arguello, H (2017) Kernel sparse subspace clustering with total variation denoising for hyperspectral remote sensing images. In: Mathematics in Imaging, 4–5. Optica Publishing Group
Zhai H, Zhang H, Xu X, Zhang L, Li P (2017) Kernel sparse subspace clustering with a spatial max pooling operation for hyperspectral remote sensing data interpretation. Remote Sens 9(4):335
Gu Y, Chanussot J, Jia X, Benediktsson JA (2017) Multiple kernel learning for hyperspectral image classification: A review. IEEE Trans Geosci Remote Sens 55(11):6547–6565
Ji, P, Zhang, T, Li, H, Salzmann, M, Reid, I (2017) Deep subspace clustering networks. Adv neural inf process syst 30
Zhou, P, Hou, Y, Feng, J (2018) Deep adversarial subspace clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1596–1604
Zhang, J, Li, C-G, You, C, Qi, X, Zhang, H, Guo, J, Lin, Z (2019) Self-supervised convolutional subspace clustering network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5473–5482
Kheirandishfard, M, Zohrizadeh, F, Kamangar, F (2020) Deep low-rank subspace clustering. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 864–865
Zeng, M, Cai, Y, Liu, X, Cai, Z, Li, X (2019) Spectral-spatial clustering of hyperspectral image based on laplacian regularized deep subspace clustering. In: IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium, 2694–2697 https://doi.org/10.1109/IGARSS.2019.8898947
Lei J, Li X, Peng B, Fang L, Ling N, Huang Q (2021) Deep spatial-spectral subspace clustering for hyperspectral image. IEEE Trans Circ Syst Video Technol 31(7):2686–2697. https://doi.org/10.1109/TCSVT.2020.3027616
Cai Y, Zeng M, Cai Z, Liu X, Zhang Z (2021) Graph regularized residual subspace clustering network for hyperspectral image clustering. Inf Sci 578:85–101
Li T, Cai Y, Zhang Y, Cai Z, Liu X (2022) Deep mutual information subspace clustering network for hyperspectral images. IEEE Geosci and Remote Sens Lett 19:1–5
Lund, B.D, Wang, T (2023) Chatting about chatgpt: how may ai and gpt impact academia and libraries? Library Hi Tech News (2023)
Gulati, A, Qin, J, Chiu, C-C, Parmar, N, Zhang, Y, Yu, J, Han, W, Wang, S, Zhang, Z, Wu, Y, et al (2020) Conformer: Convolution-augmented transformer for speech recognition. arXiv preprint arXiv:2005.08100
Kong Y, Cheng Y, Chen CLP, Wang X (2019) Hyperspectral image clustering based on unsupervised broad learning. IEEE Geosci and Remote Sens Lett 16(11):1741–1745. https://doi.org/10.1109/LGRS.2019.2907598
Liu, G, Lin, Z, Yu, Y (2010) Robust subspace segmentation by low-rank representation. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), 663–670
Funding
This work was supported by the Key Laboratory of Earth Exploration and Information Techniques of Ministry of Education, Chengdu University of Technology.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. The first draft of the manuscript was written by Zhongbiao Zhang and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding authors
Ethics declarations
Conflicts of interest
The authors declare no conflict of interest.
Additional information
Communicated by: H. Babaie.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, Z., Wang, H., Liu, S. et al. Convolution-augmented transformer network for hyperspectral image subspace clustering. Earth Sci Inform 16, 2439–2453 (2023). https://doi.org/10.1007/s12145-023-01031-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-023-01031-6