Hyperspectral pansharpening via improved PCA approach and optimal weighted fusion strategy
Introduction
Hyperspectral (HS) remote sensing image is an important area of the development of remote sensing technology. The significant characteristic of hyperspectral remote sensing data is spatial-spectral merging [1]. The spatial information and spectral information are recorded, simultaneously. Each band of the HS image is individually imaged, and each band of the HS image is superimposed to form a data cube [2]. The radiation values of one pixel on each band compose the continuous spectral information. Hyperspectral image utilizes a lot of narrow electromagnetic wave bands from the object to obtain useful information. One HS image contains hundreds or thousands of contiguous bands, and the spectral resolution can be as high as the order of nanometers [3]. Because of the rich spectral information, HS images have many relevant applications, such as the target detection [4], [5], geological exploration [6], anomaly detection [7], [8], change detection [9], and spectral unmixing [10], [11]. However, owing to the constraints of the technique and budget, the HS image usually has low spatial resolution, and cannot satisfy the requirements of applications. In contrast to the hyperspectral remote sensing sensor, panchromatic (PAN) sensor provides the PAN image with relatively higher spatial resolution. Hyperspectral pansharpening is able to obtain a fused HS image by combining the spectral information of the HS image and the spatial information of the PAN image. The fused HS image has high spectral and spatial resolution, and can improve the accuracy of hyperspectral classification [12], [13] and detection [14]. Therefore, it is necessary to fuse the HS and PAN images to generate the HS image with high spectral and spatial resolution.
In this paper, we present a new hyperspectral pansharpening method with improved PCA approach and optimal weighted fusion strategy. For the sake of overcoming the spectral distortion of the standard PCA method, a novel improved PCA approach which introduces the structural similarity (SSIM) index to select the appropriate component channel is proposed. To reduce the spatial distortion, we utilize an optimal weighted fusion strategy to extract the adequate spatial details. A injection gains matrix is generated to further control the spectral distortion, and the extracted spatial details is finally added into the interpolated HS image to obtain the fused HS image. Comparative analyses validate that the proposed method provides more spatial information compared with the state-of-the-art methods, with spectral information preserving simultaneously.
The main novelties and contributions of the proposed hyperspectral image fusion approach are listed and concluded in the following two aspects.
- 1)
To overcome the spectral distortion of the traditional PCA method, an improved PCA approach is proposed. Instead of selecting the first principal component channel as the spatial information of the HS image, the proposed method introduces the SSIM index to select the appropriate substituted component channel. The proposed method reduces the difference between the PAN image and the substituted component, and reduces the spectral distortion.
- 2)
An optimal weighted fusion strategy is proposed to obtain sufficient spatial information. Unlike the traditional methods which extract the spatial information from the PAN image, the optimal weighted fusion strategy generates the adequate spatial information from both the PAN and HS images. The weighting coefficients are adaptively obtained by minimizing an optimization equation. The proposed optimal weighted fusion strategy considers the spatial information of the HS and PAN images simultaneously, and reduces the spatial distortion.
Section snippets
Hyperspectral pansharpening
Some methods dedicated to hyperspectral pansharpening have been proposed in the past decade. These methods can be classified into four categories: component substitution (CS) approaches [15], [16], [17], [18], [19], [20], multiresolution analysis (MRA) approaches [25–27], matrix factorization based approaches [28–30], and Bayesian approaches [32–35]. The CS methods first separate the HS image into spatial and spectral information. Then the separated spatial component is substituted by the PAN
Proposed method
Fig. 1 shows the main processes of the proposed hyperspectral pansharpening method. The proposed method first presents a new improved PCA approach to obtain the spatial information of the HS image. Then the PAN image is histogram matched, and the adequate spatial details is generated from the PAN and HS images by using the optimal weighted fusion strategy. Finally, the fused HS image is obtained by generating the injection gains matrix.
Experimental results and analysis
The fusion quality of the proposed hyperspectral pansharpening method is evaluated by comparing with six state of the art algorithms which are the principal component analysis (PCA) method [17], the MTF-Generalized Laplacian Pyramid (MGLP) [26], the Guided Filter PCA (GFPCA) method [36], the transfer learning based superresolution (TLSR) method [40], the coupled nonnegative matrix factorization (CNMF) method [29], and the Bayesian Sparse (Bayesian) method [32,33].
Conclusions
In this paper, we propose a novel hyperspectral pansharpening method based on improved PCA approach and optimal weighted fusion strategy. Different from the traditional PCA method, a new improved PCA method is presented to overcome the spectral distortion. The improved PCA approach utilizes the SSIM index to generate the spatial information of the HS image. Furthermore, an optimal weighted fusion strategy is used for obtaining the sufficient spatial information from both the PAN image and the
Acknowledgments
The authors would like to thank the editors, and the anonymous reviewers for their insightful comments and suggestions which have greatly improved this paper. This work was supported in part by the National Natural Science Foundation of China (nos. 61571345, 91538101, 61501346, 61502367 and 61701360) and the 111 project (B08038). It was also partially supported by the Supported by Yangtze River Scholar Bonus Schemes of China (No. CJT160102) and the Natural Science Basic Research Plan in Shaanxi
Yunsong Li received the M.S. degree in telecommunication and information systems and the Ph.D. degree in signal and information processing from Xidian University, China, in 1999 and 2002, respectively. He joined the school of telecommunications Engineering, Xidian University in 1999 where he is currently a Professor. Prof. Li is the director of the image coding and processing center at the State Key Laboratory of Integrated Service Networks. His research interests focus on image and video
References (45)
- et al.
Hyperspectral image super-resolution using deep convolutional neural network
Neurocomputing
(2017) - et al.
A target detection method for hyperspectral image based on mixture noise model
Neurocomputing
(2016) - et al.
Hyperspectral target detection via exploiting spatial-spectral joint sparsity
Neurocomputing
(2015) - et al.
A review on spectral processing methods for geological remote sensing
Int. J. Appl. Earth Obs. Geoinf.
(2016) - et al.
Rare signal component extraction based on kernel methods for anomaly detection in hyperspectral imagery
Neurocomputing
(2013) - et al.
Hyperspectral anomaly change detection with slow feature analysis
Neurocomputing
(2015) - et al.
Hyperspectral image nonlinear unmixing and reconstruction by ELM regression ensemble
Neurocomputing
(2016) - et al.
Robust GBM hyperspectral image unmixing with superpixel segmentation based low rank and sparse representation
Neurocomputing
(2018) - et al.
Hyperspectral image classification by AdaBoost weighted composite kernel extreme learning machines
Neurocomputing
(2018) - et al.
Collaborative learning for hyperspectral image classification
Neurocomputing
(2018)
Matched shrunken subspace detectors for hyperspectral target detection
Neurocomputing
A super-resolution reconstruction algorithm for hyperspectral images
Signal Process.
Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields
IEEE Trans. Geosci. Remote Sens.
Spectral imaging for remote sensing
Linc. Lab. J.
Hyperspectral anomaly detection via a sparsity score estimation framework
IEEE Trans. Geosci. Remote Sens.
Extracting spectral contrast in Landsat thematic mapper image data using selective principal component analysis
Photogramm. Eng. Remote Sens.
A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set
Photogramm. Eng. Remote Sens.
Pansharpening based on low-rank and sparse decomposition
IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
A new look at IHS-like image fusion methods
Inf. Fusion
Improving component substitution pansharpening through multivariate regression of MS+pan data
IEEE Trans. Geosci. Remote Sens.
Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics
IEEE Trans. Geosci. Remote Sens.
Cited by (0)
Yunsong Li received the M.S. degree in telecommunication and information systems and the Ph.D. degree in signal and information processing from Xidian University, China, in 1999 and 2002, respectively. He joined the school of telecommunications Engineering, Xidian University in 1999 where he is currently a Professor. Prof. Li is the director of the image coding and processing center at the State Key Laboratory of Integrated Service Networks. His research interests focus on image and video processing, hyperspectral image processing and high-performance computing.
Jiahui Qu received B.S. degree in communication engineering from Yantai University, China, in 2014. She is currently working for a master-doctor continuous study in the State Key Laboratory of Integrated Service Networks of Xidian University. Her research interests include hyperspectral remote sensing image fusion, neural networks, and machine learning.
Wenqian Dong received B.S. degree in communication engineering from Yantai University, China, in 2014. She is currently working for a master-doctor continuous study in the State Key Laboratory of Integrated Service Networks of Xidian University. Her research interests include Compressed sensing, machine learning, and hyperspectral remote sensing image processing.
Yuxuan Zheng received the B.S. degree in School of Telecommunication Engineering of Xidian University in 2017. Now, he is currently a Ph.D. student in the State Key Laboratory of Integrated Service Networks of Xidian University. His research interests include hyperspectral image fusion, hyperspectral image super-resolution, and deep learning.