Elsevier

Neurocomputing

Volume 315, 13 November 2018, Pages 371-380
Neurocomputing

Hyperspectral pansharpening via improved PCA approach and optimal weighted fusion strategy

https://doi.org/10.1016/j.neucom.2018.07.030Get rights and content

Abstract

The purpose of hyperspectral pansharpening is to fuse the hyperspectral (HS) image and the panchromatic (PAN) image to generate an HS image with high spectral and spatial resolution. In this paper, a novel hyperspectral pansharpening method based on improved PCA approach and optimal weighted fusion strategy is proposed. First, the HS image is interpolated, and an improved PCA approach is proposed to obtain the spatial information of the HS image. To overcome the spectral distortion of the standard PCA method, the improved PCA approach utilizes the structural similarity index to select the appropriate component channel serving as the spatial information of the HS image. Subsequently, the PAN image is histogram matched with the selected component channel. In order to reduce the spatial distortion, an optimal weighted fusion strategy is presented to generate the adequate spatial details from the PAN and HS images. Finally, the injection gains matrix is generated to reduce the spectral distortion, and the fused HS image is obtained by injecting the extracted spatial details into the interpolated HS image. Experimental results demonstrate that the proposed method is superior to the state-of-the-art fusion methods in both subjective and objective evaluations.

Introduction

Hyperspectral (HS) remote sensing image is an important area of the development of remote sensing technology. The significant characteristic of hyperspectral remote sensing data is spatial-spectral merging [1]. The spatial information and spectral information are recorded, simultaneously. Each band of the HS image is individually imaged, and each band of the HS image is superimposed to form a data cube [2]. The radiation values of one pixel on each band compose the continuous spectral information. Hyperspectral image utilizes a lot of narrow electromagnetic wave bands from the object to obtain useful information. One HS image contains hundreds or thousands of contiguous bands, and the spectral resolution can be as high as the order of nanometers [3]. Because of the rich spectral information, HS images have many relevant applications, such as the target detection [4], [5], geological exploration [6], anomaly detection [7], [8], change detection [9], and spectral unmixing [10], [11]. However, owing to the constraints of the technique and budget, the HS image usually has low spatial resolution, and cannot satisfy the requirements of applications. In contrast to the hyperspectral remote sensing sensor, panchromatic (PAN) sensor provides the PAN image with relatively higher spatial resolution. Hyperspectral pansharpening is able to obtain a fused HS image by combining the spectral information of the HS image and the spatial information of the PAN image. The fused HS image has high spectral and spatial resolution, and can improve the accuracy of hyperspectral classification [12], [13] and detection [14]. Therefore, it is necessary to fuse the HS and PAN images to generate the HS image with high spectral and spatial resolution.

In this paper, we present a new hyperspectral pansharpening method with improved PCA approach and optimal weighted fusion strategy. For the sake of overcoming the spectral distortion of the standard PCA method, a novel improved PCA approach which introduces the structural similarity (SSIM) index to select the appropriate component channel is proposed. To reduce the spatial distortion, we utilize an optimal weighted fusion strategy to extract the adequate spatial details. A injection gains matrix is generated to further control the spectral distortion, and the extracted spatial details is finally added into the interpolated HS image to obtain the fused HS image. Comparative analyses validate that the proposed method provides more spatial information compared with the state-of-the-art methods, with spectral information preserving simultaneously.

The main novelties and contributions of the proposed hyperspectral image fusion approach are listed and concluded in the following two aspects.

  • 1)

    To overcome the spectral distortion of the traditional PCA method, an improved PCA approach is proposed. Instead of selecting the first principal component channel as the spatial information of the HS image, the proposed method introduces the SSIM index to select the appropriate substituted component channel. The proposed method reduces the difference between the PAN image and the substituted component, and reduces the spectral distortion.

  • 2)

    An optimal weighted fusion strategy is proposed to obtain sufficient spatial information. Unlike the traditional methods which extract the spatial information from the PAN image, the optimal weighted fusion strategy generates the adequate spatial information from both the PAN and HS images. The weighting coefficients are adaptively obtained by minimizing an optimization equation. The proposed optimal weighted fusion strategy considers the spatial information of the HS and PAN images simultaneously, and reduces the spatial distortion.

Section snippets

Hyperspectral pansharpening

Some methods dedicated to hyperspectral pansharpening have been proposed in the past decade. These methods can be classified into four categories: component substitution (CS) approaches [15], [16], [17], [18], [19], [20], multiresolution analysis (MRA) approaches [25–27], matrix factorization based approaches [28–30], and Bayesian approaches [32–35]. The CS methods first separate the HS image into spatial and spectral information. Then the separated spatial component is substituted by the PAN

Proposed method

Fig. 1 shows the main processes of the proposed hyperspectral pansharpening method. The proposed method first presents a new improved PCA approach to obtain the spatial information of the HS image. Then the PAN image is histogram matched, and the adequate spatial details is generated from the PAN and HS images by using the optimal weighted fusion strategy. Finally, the fused HS image is obtained by generating the injection gains matrix.

Experimental results and analysis

The fusion quality of the proposed hyperspectral pansharpening method is evaluated by comparing with six state of the art algorithms which are the principal component analysis (PCA) method [17], the MTF-Generalized Laplacian Pyramid (MGLP) [26], the Guided Filter PCA (GFPCA) method [36], the transfer learning based superresolution (TLSR) method [40], the coupled nonnegative matrix factorization (CNMF) method [29], and the Bayesian Sparse (Bayesian) method [32,33].

Conclusions

In this paper, we propose a novel hyperspectral pansharpening method based on improved PCA approach and optimal weighted fusion strategy. Different from the traditional PCA method, a new improved PCA method is presented to overcome the spectral distortion. The improved PCA approach utilizes the SSIM index to generate the spatial information of the HS image. Furthermore, an optimal weighted fusion strategy is used for obtaining the sufficient spatial information from both the PAN image and the

Acknowledgments

The authors would like to thank the editors, and the anonymous reviewers for their insightful comments and suggestions which have greatly improved this paper. This work was supported in part by the National Natural Science Foundation of China (nos. 61571345, 91538101, 61501346, 61502367 and 61701360) and the 111 project (B08038). It was also partially supported by the Supported by Yangtze River Scholar Bonus Schemes of China (No. CJT160102) and the Natural Science Basic Research Plan in Shaanxi

Yunsong Li received the M.S. degree in telecommunication and information systems and the Ph.D. degree in signal and information processing from Xidian University, China, in 1999 and 2002, respectively. He joined the school of telecommunications Engineering, Xidian University in 1999 where he is currently a Professor. Prof. Li is the director of the image coding and processing center at the State Key Laboratory of Integrated Service Networks. His research interests focus on image and video

References (45)

  • WangZ.Y. et al.

    Matched shrunken subspace detectors for hyperspectral target detection

    Neurocomputing

    (2018)
  • ZhangH. et al.

    A super-resolution reconstruction algorithm for hyperspectral images

    Signal Process.

    (2012)
  • LiJ. et al.

    Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields

    IEEE Trans. Geosci. Remote Sens.

    (2012)
  • H.K. Burke et al.

    Spectral imaging for remote sensing

    Linc. Lab. J.

    (2003)
  • ZhaoR. et al.

    Hyperspectral anomaly detection via a sparsity score estimation framework

    IEEE Trans. Geosci. Remote Sens.

    (2017)
  • P.S. Chavez et al.

    Extracting spectral contrast in Landsat thematic mapper image data using selective principal component analysis

    Photogramm. Eng. Remote Sens.

    (1989)
  • V.K. Shettigara

    A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set

    Photogramm. Eng. Remote Sens.

    (1992)
  • RongK. et al.

    Pansharpening based on low-rank and sparse decomposition

    IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.

    (2014)
  • TuT.M. et al.

    A new look at IHS-like image fusion methods

    Inf. Fusion

    (2001)
  • C. Laben and B. Brower, Process for Enhacing the Spatial Resolution of Multispectral Imagery using Pan-sharpening, U.S....
  • B. Aiazzi et al.

    Improving component substitution pansharpening through multivariate regression of MS+pan data

    IEEE Trans. Geosci. Remote Sens.

    (2007)
  • C. Thomas et al.

    Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics

    IEEE Trans. Geosci. Remote Sens.

    (2008)
  • Cited by (0)

    Yunsong Li received the M.S. degree in telecommunication and information systems and the Ph.D. degree in signal and information processing from Xidian University, China, in 1999 and 2002, respectively. He joined the school of telecommunications Engineering, Xidian University in 1999 where he is currently a Professor. Prof. Li is the director of the image coding and processing center at the State Key Laboratory of Integrated Service Networks. His research interests focus on image and video processing, hyperspectral image processing and high-performance computing.

    Jiahui Qu received B.S. degree in communication engineering from Yantai University, China, in 2014. She is currently working for a master-doctor continuous study in the State Key Laboratory of Integrated Service Networks of Xidian University. Her research interests include hyperspectral remote sensing image fusion, neural networks, and machine learning.

    Wenqian Dong received B.S. degree in communication engineering from Yantai University, China, in 2014. She is currently working for a master-doctor continuous study in the State Key Laboratory of Integrated Service Networks of Xidian University. Her research interests include Compressed sensing, machine learning, and hyperspectral remote sensing image processing.

    Yuxuan Zheng received the B.S. degree in School of Telecommunication Engineering of Xidian University in 2017. Now, he is currently a Ph.D. student in the State Key Laboratory of Integrated Service Networks of Xidian University. His research interests include hyperspectral image fusion, hyperspectral image super-resolution, and deep learning.

    View full text