Xijuan Zhang
ABSTRACT
Most of the existing hyperspectral image change detection methods based on deep learning utilize convolutional neural networks (CNN), recurrent neural networks (RNN), and long-short-term memory (LSTM). However, CNNs primarily focus on extracting spatial information from hyperspectral images, thereby failing to fully utilize channel and temporal information. In this paper, we propose a network called STSTNet that can jointly extract spatial, spectral, and temporal feature information. To achieve a richer feature representation space, we employ a spatial attention module to extract spatial feature information from feature maps, and a spectral transformer encoder structure to extract abundant spectral sequence information. Specifically, we design two temporal information extraction structures after each spatial information extraction structure or spectral information extraction structure to capture correlations and interactions within the hyperspectral images. Extensive experiments have been conducted on three hyperspectral datasets, and the experimental results demonstrate the network’s excellent performance.
- L. Bruzzone and D.F. Prieto. 2000. Automatic analysis of the difference image for unsupervised change detection. IEEE Transactions on Geoscience and Remote Sensing 38, 3 (2000), 1171–1182. https://doi.org/10.1109/36.843009Google Scholar
Cross Ref
- Osmar A Carvalho Júnior, Renato F Guimarães, Alan R Gillespie, Nilton C Silva, and Roberto AT Gomes. 2011. A new approach to change vector analysis using distance and similarity measures. Remote Sensing 3, 11 (2011), 2473–2493.Google ScholarCross Ref
- Hongruixuan Chen, Chen Wu, Bo Du, Liangpei Zhang, and Le Wang. 2019. Change detection in multisource VHR images via deep siamese convolutional multiple-layers recurrent neural network. IEEE Transactions on Geoscience and Remote Sensing 58, 4 (2019), 2848–2864.Google ScholarCross Ref
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).Google Scholar
- Emmanuel Dufourq and Bruce A Bassett. 2017. Eden: Evolutionary deep networks for efficient machine learning. In 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech). IEEE, 110–115.Google Scholar
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.Google ScholarDigital Library
- Lazhar Khelifi and Max Mignotte. 2020. Deep learning for change detection in remote sensing images: Comprehensive review and meta-analysis. Ieee Access 8 (2020), 126385–126400.Google ScholarCross Ref
- Yun Lin, Shutao Li, Leyuan Fang, and Pedram Ghamisi. 2019. Multispectral change detection with bilinear convolutional neural networks. IEEE Geoscience and Remote Sensing Letters 17, 10 (2019), 1757–1761.Google ScholarCross Ref
- Xuejun Mao and Hanhuai Pan. 2013. An Improved DOA Estimation Algorithm Based on Wavelet Operator.J. Commun. 8, 12 (2013), 839–844.Google ScholarCross Ref
- Lichao Mou, Lorenzo Bruzzone, and Xiao Xiang Zhu. 2018. Learning spectral-spatial-temporal features via a recurrent convolutional neural network for change detection in multispectral imagery. IEEE Transactions on Geoscience and Remote Sensing 57, 2 (2018), 924–935.Google ScholarCross Ref
- Allan A Nielsen, Knut Conradsen, and James J Simpson. 1998. Multivariate alteration detection (MAD) and MAF postprocessing in multispectral, bitemporal image data: New approaches to change detection studies. Remote Sensing of Environment 64, 1 (1998), 1–19.Google ScholarCross Ref
- Salvatore Resta, Nicola Acito, Marco Diani, Giovanni Corsini, Thomas Opsahl, and Trym Vegard Haavardsholm. 2011. Detection of small changes in airborne hyperspectral imagery: Experimental results over urban areas. In 2011 6th International Workshop on the Analysis of Multi-temporal Remote Sensing Images (Multi-Temp). 5–8. https://doi.org/10.1109/Multi-Temp.2011.6005033Google ScholarCross Ref
- Ahram Song, Jaewan Choi, Youkyung Han, and Yongil Kim. 2018. Change detection in hyperspectral images using recurrent 3D fully convolutional networks. Remote Sensing 10, 11 (2018), 1827.Google ScholarCross Ref
- Ruoxi Song, Weihan Ni, Wei Cheng, and Xianghai Wang. 2022. CSANet: Cross-temporal interaction symmetric attention network for hyperspectral image change detection. IEEE Geoscience and Remote Sensing Letters 19 (2022), 1–5.Google ScholarCross Ref
- Qi Wang, Zhenghang Yuan, Qian Du, and Xuelong Li. 2018. GETNET: A general end-to-end 2-D CNN framework for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing 57, 1 (2018), 3–13.Google ScholarCross Ref
- Yanheng Wang, Danfeng Hong, Jianjun Sha, Lianru Gao, Lian Liu, Yonggang Zhang, and Xianhui Rong. 2022. Spectral–spatial–temporal transformers for hyperspectral image change detection. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–14.Google Scholar
- Zhengwei Yang and Rick Mueller. 2007. Spatial-spectral cross-correlation for change detection: a case study for citrus coverage change detection. In ASPRS 2007 Annual conference, Vol. 2. 767–777.Google Scholar
- Hui Zhang, Maoguo Gong, Puzhao Zhang, Linzhi Su, and Jiao Shi. 2016. Feature-level change detection using deep representation and feature change analysis for multispectral imagery. IEEE Geoscience and Remote Sensing Letters 13, 11 (2016), 1666–1670.Google ScholarCross Ref
- Li Zhang, Ding Wang, and Ying Wu. 2014. Performance Analysis of TDOA and FDOA Location by Differential Calibration with Calibration Sources.J. Commun. 9, 6 (2014), 483–489.Google ScholarCross Ref
Index Terms
- Change Detection in Hyperspectral Images based on a Spatial-Spectral-Temporal Attention Network
Recommendations
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
AbstractRecently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Object-level Hyperspectral Target Detection Based on Spectral-Spatial Features Integrated YOLOv4-Tiny Network
IVSP '22: Proceedings of the 2022 4th International Conference on Image, Video and Signal ProcessingThe spectral resolution and spatial resolution of hyperspectral remote sensing images are mutually limited. To keep the same signal-to-noise ratio, the spatial resolution will decrease when the spectral resolution improves. The targets in low-resolution ...
A spatial-spectral SIFT for hyperspectral image matching and classification
Highlights- We propose a new processing and analysis model called unified model of spectral value and gradient change(UMSGC) for HSIs.
AbstractThe scale-invariant feature transform (SIFT) is known as one of the most robust local invariant feature and is widely applied to image matching and classification. However, There is few studies on SIFT for hyperspectral image (HSI). ...
Comments