Joint Classification of Multi-Source Remote Sensing Data Based on Multi-Scale Features and Attention Mechanism
Authors: 
Beibei Zhang, Tao Zhang, Yu Zhang, Duo Zhao, Zhen YeAuthors Info & Claims
Beibei Zhang, Tao Zhang, Yu Zhang, Duo Zhao, Zhen YeAuthors Info & ClaimsAbstract
In order to fully extract spatial and elevation information for joint classification of hyperspectral image (HSI) and LiDAR (LiDAR) data, a multi-source remote sensing classification network based on multi-scale feature and attention mechanism is proposed in this paper. Firstly, a multi-scale extraction module is designed to extract spatial features of HSI and LiDAR data. Then, the weights of HSI and LiDAR spatial features are assigned adaptively by employing a spatial attention module. Next, the feature maps from two branches are fused, and the final classification results are yielded by Softmax. Experiments conducted on two public datasets demonstrate that the proposed method outperforms several existing advanced methods for HSI and LiDAR joint classification.
References
[1]
Salem F., Kafatos M., El‐Ghazawi T., Smith W.L., Shabanov N. V., Huang C., Sun G., Tesfaye M. Z., Prasad S. Hyperspectral image assessment of oil‐contaminated wetland. International Journal of Remote Sensing, 2005, 26(4): 811-821.
[2]
Dale L. M., Thewis A., Boudry C., Despagne F., Gérard P., Lebeau F., Massart X., Sindic M. Hyperspectral imaging applications in agriculture and agro-food product quality and safety control: A review. Applied Spectroscopy Reviews, 2013, 48(2): 142-159.
[3]
Song A., Kim Y. Deep learning-based hyperspectral image classification with application o environmental geographic information systems. Korean Journal of Remote Sensing, 2017, 33(62): 1061-1073.
[4]
Briottet X., Boucher Y., Dimmeler A., Reulke R., Suomalainen J., Mueller A., Holzwarth S., Strobl P., Bulatov D., Richter R., Schlapfer D. Military applications of hyperspectral imagery Targets and backgrounds XII: Characterization and representation. SPIE, 2006, 6239: 82-89.
[5]
Chen, Y., Li, X., & Hu, W. Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014,7(6), 2094-2107.
[6]
Li, W., Hu, Q., & Li, X. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. Remote Sensing, 2017, 9(7), 673.
[7]
Hu X., Li Y., Shan J., Gong J., Li L., Gong Y. Road centerline extraction in complex urban scenes from LiDAR data based on multiple features. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(11): 7448-7456.
[8]
Anttila K., Hakala T., Kaasalainen S., Kaartinen H., Kukko A., Lehtola V., Mäkynen J., Nevalainen O., Rosnell T., Sarjakoski T., Silvennoinen R., Virtanen J. Calibrating laser scanner data from snow surfaces: Correction of intensity effects. Cold Regions Science and Technology, 2016, 121: 52-59.
[9]
Ghamisi P., Rasti B., Yokoya N., Benediktsson J. A., Waske B., Heremans R., Zhu X. X., Liang X., Moser G., Cheng G., Liao W., Plaza A., Tuia D., Pelletier C., Shi Q., Yokoya N., Zhu X. X. Multisource and multitemporal data fusion in remote sensing: A comprehensive review of the state of the art. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(1): 6-39.
[10]
Debes C., Merentitis A., Heremans R., Bovolo F., Bruzzone L., Camps-Valls G., Chan J. C. W., Dumitru C. O., Gautama S., Ghamisi P., Kang X., Li W., Li Y., Liao W., Moser G., Palsson O. P., Plaza A., Rasti B., Sun Y., Tuia D., Yokoya N., Zhu X. X. Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2405-2418.
[11]
Hong D., Gao L., Hang R., Yuan Y., Zhu Y., Zhang C., Zhang H., Huang J., Zhang L. Deep encoder–decoder networks for classification of hyperspectral and LiDAR data. IEEE Geoscience and Remote Sensing Letters, 2020, 19: 1-5.
[12]
Cao X., Zhou F., Xu L., Liu Q., Zhang L., Li Y. Hyperspectral image classification with Markov random fields and a convolutional neural network. IEEE Transactions on image processing, 2018, 27(5): 2354-2367.
[13]
Xu X., Li W., Ran Q., Du P., Liu X., Zhang Y. Multisource remote sensing data classification based on convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(2): 937-949.
[14]
Zhang, X., Li, X., & Liu, Y. Hyperspectral image classification based on multi-scale features. Journal of Sensors, 2017, 2017.
[15]
Xu, X., Zhang, L., Zhang, D., Zhu X., Wu X., Liang D. SE-Res2Net: A novel multi-scale feature extraction module based on Res2Net and SE-Net for hyperspectral image classification. Remote Sensing, 2020, 12(4), 663.
[16]
Yang, Y., Zhou, Y., & Chen, J. Hyperspectral image classification based on multi-scale hybrid convolutional
[17]
Hang R., Li Z., Ghamisi P., Benediktsson J. A., Zhu X., Zhang H., Huang J., Zhang L. Classification of hyperspectral and LiDAR data using coupled CNNs. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7): 4939-4950.
[18]
Xi Y., Ye Z. Multi-Scale Feature Extraction for Joint Classification of Hyperspectral and LiDAR Data. Journal of Beijing Institute of Technology, 2023, 32(01): 13-22.
[19]
Gader P., Zare A., Close R. Muufl gulfport hyperspectral and lidar airborne data set. Univ. Florida, Gainesville, FL, USA, Tech. Rep. REP-2013-570, 2013.
[20]
Rasti B., Ghamisi P., Gloaguen R. Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3997-4007.
Index Terms
- Joint Classification of Multi-Source Remote Sensing Data Based on Multi-Scale Features and Attention Mechanism
Recommendations
Parameterizing the aerodynamic roughness length on a regional scale based on multi-source remote-sensing data
Aerodynamic roughness length z0 is one of those important biophysical parameters that influence energy exchange at the land–atmosphere interface, so it is significant to quantify the z0 accurately. In this article, a scheme parameterizing land-surface z...
Remote sensing of seagrasses in a patchy multi-species environment
We tested the utility of IKONOS satellite imagery to map seagrass distribution and biomass in a 4.1 km2 area around Chumbe Island, Zanzibar, Tanzania. Considered to be a challenging environment to map, this area is characterized by a diverse mix of ...
Comments
Information & Contributors
Information
Published In
November 2023
237 pages
ISBN:9798400709272
DOI:10.1145/3638682
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
VSIP 2023
VSIP 2023: 2023 the 5th International Conference on Video, Signal and Image Processing
November 24 - 26, 2023
Harbin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 18Total Downloads
- Downloads (Last 12 months)18
- Downloads (Last 6 weeks)1
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format