Abstract
Hyperspectral Images (HSI) are commonly used for classification thanks to their rich spectral feature information along with their spatial feature information. Convolutional Neural Network (CNN) based deep learning methods are commonly used in HSI classification (HSIC) applications to process the high nonlinearity and high dimensionality of HSI. This study proposes a method consisting of a combination of multipath Hybrid CNN and a Squeeze and Excitation (SE) network for HSIC. Features extracted with different kernel sizes in the multipath method are used together to extract richer feature information from HSI in this proposed method (PM). In the Hybrid CNN used in PM, 3D CNN was used to extract the spectral-spatial features. However, computational complexity increases with 3D CNN. Computational complexity is decreased with the use of Hybrid CNN. In addition, 2D CNN used in Hybrid CNN provides more spatial feature information to be extracted. However, in this study, 2D depthwise separable convolution (DSC) was used instead of 2D CNN. By using 2D DSC instead of standard 2D CNN, computational cost and the number of trainable parameters is significantly decreased. Finally, the PM is combined with the SE network to advance the HSIC accuracies. The SE network is designed to enhance the representation quality of CNN. WHU-Hi-HongHu (WHHH), WHU-Hi-HanChuan (WHHC), and WHU-Hi-LongKou (WHLK) datasets were used to evaluate the classification accuracies of the PM. Using a 5% training sample with WHLK, WHHC and WHHH, OA values of 99.86%, 97.51% and 97.64% were obtained. Furthermore, the PM was compared with the latest technology methods in the literature and outperformed all methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Not applicable.
References
Ahmad M, Khan A, Khan AM et al (2019) Spatial prior fuzziness pool-based interactive classification of hyperspectral images. Remote Sens 11:1–19. https://doi.org/10.3390/rs11091136
Ahmad M, Khan AM, Mazzara M, et al (2020) A fast and compact 3-D CNN for hyperspectral ımage classification. IEEE Geosci Remote Sens Lett 1–5. https://doi.org/10.1109/LGRS.2020.3043710
Alcolea A, Paoletti ME, Haut JM et al (2020) Inference in supervised spectral classifiers for on-board hyperspectral imaging: An overview. Remote Sens 12:1–29. https://doi.org/10.3390/rs12030534
Ben Hamida A, Benoit A, Lambert P, Ben Amar C (2018) 3-D deep learning approach for remote sensing image classification. IEEE Trans Geosci Remote Sens 56:4420–4434. https://doi.org/10.1109/TGRS.2018.2818945
Blanzieri E, Melgani F (2008) Nearest neighbor classification of remote sensing images with the maximal margin principle. IEEE Trans Geosci Remote Sens 46:1804–1811. https://doi.org/10.1109/TGRS.2008.916090
Chen Y, Zhao X, Jia X (2015) Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE J Sel Top Appl Earth Obs Remote Sens 8:2381–2392. https://doi.org/10.1109/JSTARS.2015.2388577
Cheng G, Li Z, Han J et al (2018) Exploring Hierarchical Convolutional Features for Hyperspectral Image Classification. IEEE Trans Geosci Remote Sens 56:6712–6722. https://doi.org/10.1109/TGRS.2018.2841823
Dang L, Pang P, Lee J (2020) Depth-wise separable convolution neural network with residual connection for hyperspectral image classification. Remote Sens 12:1–20. https://doi.org/10.3390/rs12203408
Data H, Chen Y, Lin Z et al (2015) Deep Learning-Based Classification of Hyperspectral Data. IEEE J Sel Top Appl Earth Obs Remote Sens 7:1–14
Ding Y, Zhang Z, Zhao X et al (2022) Multi-feature fusion: Graph neural network and CNN combining for hyperspectral image classification. Neurocomputing 501:246–257. https://doi.org/10.1016/j.neucom.2022.06.031
Firat H, Asker ME, Bayindir Mİ, Hanbay D (2022a) 3D residual spatial–spectral convolution network for hyperspectral remote sensing image classification. Neural Comput Appl 8:. https://doi.org/10.1007/s00521-022-07933-8
Firat H, Asker ME, Hanbay D (2022b) Classification of hyperspectral remote sensing images using different dimension reduction methods with 3D/2D CNN. Remote Sens Appl Soc Environ 100694. https://doi.org/10.1016/j.rsase.2022.100694
Firat H, Hanbay D (2021) Classification of hyperspectral images using 3D CNN based ResNet50. SIU 2021 - 29th IEEE Conf Signal Process Commun Appl Proc 6–9. https://doi.org/10.1109/SIU53274.2021.9477899
Firat H, Hanbay D (2022) 3 Boyutlu Evrişimsel Sinir Ağı Kullanılarak Hiperspektral Görüntülerin Sınıflandırılması Classification of Hyperspectral Images Using 3D Convolutional Neural Network. Türk Doğa Ve Fen Derg 11:19–28
Firat H, Hanbay D (2023) Comparison of 3D CNN based deep learning architectures using hyperspectral images. J Fac Eng Archit Gazi Univ 38:521–534. https://doi.org/10.17341/gazimmfd.977688
Fırat H, Asker ME, Ilyas M, Hanbay D (2022a) Spatial-spectral classification of hyperspectral remote sensing images using 3D CNN based LeNet-5 architecture. Infrared Phys Technol 127:. https://doi.org/10.1016/j.infrared.2022.104470
Fırat H, Emin M, Mehmet A, et al (2022b) Hybrid 3D / 2D Complete Inception Module and Convolutional Neural Network for Hyperspectral Remote Sensing Image Classification. Neural Process Lett 1–44. https://doi.org/10.1007/s11063-022-10929-z
Fırat H, Hanbay D (2022) 4CF-Net: New 3D convolutional neural network for spectral spatial classification of hyperspectral remote sensing images. J Fac Eng Archit Gazi Univ 37:439–453. https://doi.org/10.17341/gazimmfd.901291
Gao H, Chen Z, Li C (2021) Sandwich convolutional neural network for hyperspectral image classification using spectral feature enhancement. IEEE J Sel Top Appl Earth Obs Remote Sens 14:3006–3015. https://doi.org/10.1109/JSTARS.2021.3062872
Gong H, Li Q, Li C, et al (2021) Multiscale ınformation fusion for hyperspectral ımage classification based on hybrid 2D-3D CNN. Remote Sens 13:. https://doi.org/10.3390/rs13122268
Han Y, Wei C, Zhou R, et al (2020) Combining 3D-CNN and squeeze-and-excitation networks for remote sensing sea ıce ımage classification. Math Probl Eng 2020:. https://doi.org/10.1155/2020/8065396
Hanbay K (2020) Hyperspectral image classification using convolutional neural network and two-dimensional complex Gabor transform. J Fac Eng Archit Gazi Univ 35:443–456. https://doi.org/10.17341/gazimmfd.479086
Hong D, Han Z, Yao J, et al (2022) SpectralFormer: rethinking hyperspectral ımage classification with transformers. IEEE Trans Geosci Remote Sens 60:. https://doi.org/10.1109/TGRS.2021.3130716
Hörig B, Kühn F, Oschütz F, Lehmann F (2001) HyMap hyperspectral remote sensing to detect hydrocarbons. Int J Remote Sens 22:1413–1422. https://doi.org/10.1080/01431160120909
Huang J, He H, Lv R et al (2022) Non-destructive detection and classification of textile fibres based on hyperspectral imaging and 1D-CNN. Anal Chim Acta 1224:340238. https://doi.org/10.1016/j.aca.2022.340238
Iyer P, A S, Lal S (2021) Deep learning ensemble method for classification of satellite hyperspectral images. Remote Sens Appl Soc Environ 23:100580. https://doi.org/10.1016/j.rsase.2021.100580
Jia J, Wang Y, Chen J et al (2020) Status and application of advanced airborne hyperspectral imaging technology: A review. Infrared Phys Technol 104:103115. https://doi.org/10.1016/j.infrared.2019.103115
Kang X, Duan P, Li S (2020) Hyperspectral image visualization with edge-preserving filtering and principal component analysis. Inf Fusion 57:130–143. https://doi.org/10.1016/j.inffus.2019.12.003
Karadağ B, Arı A, Karadağ M (2021) Derin Öğrenme modellerinin sinirsel stil aktarımı performanslarının karşılaştırılması. J Polytech 0900:1611–1622. https://doi.org/10.2339/politeknik.885838
Lanthier Y, Bannari A, Haboudane D et al (2008) Hyperspectral data segmentation and classification in precision agriculture: A multi-scale analysis. Int Geosci Remote Sens Symp 2:585–588. https://doi.org/10.1109/IGARSS.2008.4779060
Lee J, Kim Y, Jeong M, et al (2018) 3D convolutional neural networks for soccer object motion recognition. Int Conf Adv Commun Technol ICACT 2018-Febru:354–358. https://doi.org/10.23919/ICACT.2018.8323754
Li S, Zhang K, Hao Q et al (2018) Hyperspectral anomaly detection with multiscale attribute and edge-preserving filters. IEEE Geosci Remote Sens Lett 15:1605–1609. https://doi.org/10.1109/LGRS.2018.2853705
Li W, Wu G, Zhang F, Du Q (2017) Hyperspectral image classification using deep pixel-pair features. IEEE Trans Geosci Remote Sens 55:844–853. https://doi.org/10.1109/TGRS.2016.2616355
Lin Z, Ji K, Leng X, Kuang G (2019) Squeeze and excitation rank faster R-CNN for ship detection in SAR images. IEEE Geosci Remote Sens Lett 16:751–755. https://doi.org/10.1109/LGRS.2018.2882551
Liu X, Yu J, Kurihara T et al (2022) Hyperspectral imaging for green pepper segmentation using a complex-valued neural network. Optik (stuttg) 265:169527. https://doi.org/10.1016/j.ijleo.2022.169527
Lu G, Zhang W, Wang Z (2022) Optimizing depthwise separable convolution operations on GPUs. IEEE Trans Parallel Distrib Syst 33:70–87. https://doi.org/10.1109/TPDS.2021.3084813
Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. Int Geosci Remote Sens Symp 2015-Novem:4959–4962. https://doi.org/10.1109/IGARSS.2015.7326945
Melgani F, Bruzzone L (2004) Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans Geosci Remote Sens 42:1778–1790. https://doi.org/10.1109/TGRS.2004.831865
Mohan A, Venkatesan M (2020) HybridCNN based hyperspectral image classification using multiscale spatiospectral features. Infrared Phys Technol 108:. https://doi.org/10.1016/j.infrared.2020.103326
Mou L, Ghamisi P, Zhu XX (2017) Deep recurrent neural networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55:3639–3655. https://doi.org/10.1109/TGRS.2016.2636241
Mughees A, Tao L (2017) Efficient deep auto-encoder learning for the classification of hyperspectral images. Proc - 2016 Int Conf Virtual Real Vis ICVRV 2016 44–51. https://doi.org/10.1109/ICVRV.2016.16
Okwuashi O, Ndehedehe CE (2020) Deep support vector machine for hyperspectral image classification. Pattern Recognit 103:107298. https://doi.org/10.1016/j.patcog.2020.107298
Paoletti ME, Haut JM, Plaza J, Plaza A (2018) A new deep convolutional neural network for fast hyperspectral image classification. ISPRS J Photogramm Remote Sens 145:120–147. https://doi.org/10.1016/j.isprsjprs.2017.11.021
Rajendran T, Valsalan P, Amutharaj J, et al (2022) Hyperspectral ımage classification model using squeeze and excitation network with deep learning. Comput Intell Neurosci 2022:. https://doi.org/10.1155/2022/9430779
Ratle F, Camps-Valls G, Weston J (2010) Semisupervised neural networks for efficient hyperspectral image classification. IEEE Trans Geosci Remote Sens 48:2271–2282. https://doi.org/10.1109/TGRS.2009.2037898
Roy SK, Chatterjee S, Bhattacharyya S et al (2020a) Lightweight spectral-spatial squeeze-and- excitation residual bag-of-features learning for hyperspectral classification. IEEE Trans Geosci Remote Sens 58:5277–5290. https://doi.org/10.1109/TGRS.2019.2961681
Roy SK, Dubey SR, Chatterjee S, Chaudhuri BB (2020b) FuSENet: Fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification. IET Image Process 14:1653–1661. https://doi.org/10.1049/iet-ipr.2019.1462
Roy SK, Krishna G, Dubey SR, Chaudhuri BB (2019) HybridSN: exploring 3D-2D CNN feature hierarchy for hyperspectral ımage classification. arXiv 17:277–281
Roy SK, Manna S, Song T, Bruzzone L (2020c) Attention-based adaptive spectral-spatial kernel resnet for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59:7831–7843. https://doi.org/10.1109/TGRS.2020.3043267
Song W, Li S, Fang L (2018) Hyperspectral image classification with deep feature fusion network. IEEE Trans Geosci Remote Sens 99:3173–3184. https://doi.org/10.1109/IGARSS.2019.8898520
Üzen H, Turkoglu M, Aslan M, Hanbay D (2022) Depth-wise squeeze and excitation block-based efficient-unet model for surface defect detection. Vis Comput. https://doi.org/10.1007/s00371-022-02442-0
Uzen H, Turkoglu M, Hanbay D (2021) Texture defect classification with multiple pooling and filter ensemble based on deep neural network. Expert Syst Appl 175:114838. https://doi.org/10.1016/j.eswa.2021.114838
Wang Y, Yu W, Fang Z (2020) Multiple Kernel-based SVM classification of hyperspectral images by combining spectral, spatial, and semantic information. Remote Sens 12:. https://doi.org/10.3390/RS12010120
Xu H, Zhang H, He W, Zhang L (2019) Superpixel-based spatial-spectral dimension reduction for hyperspectral imagery classification. Neurocomputing 360:138–150. https://doi.org/10.1016/j.neucom.2019.06.023
Yang X, Ye Y, Li X et al (2018) Hyperspectral image classification with deep learning models. IEEE Trans Geosci Remote Sens 56:5408–5423. https://doi.org/10.1109/TGRS.2018.2815613
Zhang M, Li W, Du Q (2018) Diverse region-based CNN for hyperspectral image classification. IEEE Trans Image Process 27:2623–2634. https://doi.org/10.1109/TIP.2018.2809606
Zhong Y, Hu X, Luo C et al (2020) WHU-Hi: UAV-borne hyperspdectral with high spatial resolution (H2) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF. Remote Sens Environ 250:112012. https://doi.org/10.1016/j.rse.2020.112012
Zhou F, Hang R, Liu Q, Yuan X (2019) Hyperspectral image classification using spectral-spatial LSTMs. Neurocomputing 328:39–47. https://doi.org/10.1016/j.neucom.2018.02.105
Zhu L, Chen Y, Ghamisi P, Benediktsson JA (2018) Generative adversarial networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 56:5046–5063. https://doi.org/10.1109/TGRS.2018.2805286
Author information
Authors and Affiliations
Contributions
Author 1: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing—review & editing.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by: H. Babaie
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ari, A. Multipath feature fusion for hyperspectral image classification based on hybrid 3D/2D CNN and squeeze-excitation network. Earth Sci Inform 16, 175–191 (2023). https://doi.org/10.1007/s12145-022-00929-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-022-00929-x