Robust Hyperspectral Image Pan-Sharpening via Channel-Constrained Spatial Spectral Network

Abstract

Hyperspectral image (HSI) pan-sharpening, in which high-resolution (HR) spatial details from multispectral image (MSI) are employed to enhance the spatial resolution of HSI, has recently attracted much attention. Convolutional neural network (CNN) extracts more comprehensive features and is proved to be effective in pan-sharpening. However, the conventional CNN model becomes less efficient in handling noisy LR-HSI. Especially, most of the existing CNN based methods treat low-resolution (LR) HSI with abundant low-frequency information equally across channels, hence hindering the representation ability of CNN. To address this problem, we propose a robust channel-constrained spatial spectral network (RCSSN). Specifically, it is formed by two robust channel-constrained blocks (RCB) with short skip connections. Furthermore, we conjoin the traditional mean square error (MSE) loss and the first-order derivative feature error (FODFE) loss together to learn network parameters, which enables the network to suppress the effect of noise on image edge and texture. Both the quantitative assessment and the visual assessment results confirm that the proposed network yields HR-HSI that are superior to the images obtained by the compared state-of-the-art methods.

This work was supported in part by the National Natural Science Foundation of China under grant 61702169, the Natural Science Foundation of Hunan Province under Grant 2018JJ3070, the National Natural Science Fund of China for International Cooperation and Exchanges under Grant 61520106001, and the Fundamental Research Funds for the Central Universities under Grant 531107050878. The authors would like to thank Prof. Shutao Li and Hui Lin in the Hunan University for their consistent and illuminating encouragement and guidance to this work.