Abstract
Recently, many methods have been proposed to generate a high spatial resolution (HR) hyperspectral image (HSI) by fusing HSI and multispectral image (MSI). Most methods need a precondition that HSI and MSI are well registered. However, in practice, it is hard to acquire registered HSI and MSI. In this paper, a synchronous nonnegative matrix factorization (SNMF) is proposed to directly fuse unregistered HSI and MSI. The proposed SNMF does not require the registration operation by modeling the abundances of unregistered HSI and MSI independently. Moreover, to exploit both HSI and MSI in the endmember optimization of the desired HR HSI, the unregistered HSI and MSI fusion is formulated as a bound-constrained optimization problem. A synchronous projected gradient method is proposed to solve this bound-constrained optimization problem. Experiments on both simulated and real data demonstrate that the proposed SNMF outperforms the state-of-the-art methods.
The first author is a PhD student.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Sun, H., Zheng, X., Lu, X., Wu, S.: Spectral-spatial attention network for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 58(5), 3232–3245 (2020)
Yokoya, N., Grohnfeldt, C., Chanussot, J.: Hyperspectral and multispectral data fusion: a comparative review of the recent literature. IEEE Geosci. Remote Sens. Mag. 5(2), 29–56 (2017)
Sun, H., Li, S., Zheng, X., Lu, X.: Remote sensing scene classification by gated bidirectional network. IEEE Trans. Geosci. Remote Sens. 58(1), 82–96 (2020)
Lu, X., Yuan, Y., Zheng, X.: Joint dictionary learning for multispectral change detection. IEEE Trans. Cybern. 47(4), 884–897 (2017)
Zhang, K., Wang, M., Yang, S., Jiao, L.: Spatial-spectral-graph-regularized low-rank tensor decomposition for multispectral and hyperspectral image fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 11(4), 1030–1040 (2018)
Gao, D., Zhentao, H., Ye, R.: Self-dictionary regression for hyperspectral image super-resolution. Remote Sens. 10(10), 1574 (2018)
Akhtar, N., Shafait, F., Mian, A.: Bayesian sparse representation for hyperspectral image super resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3631–3640, June 2015
Borsoi, R.A., Imbiriba, T., Bermudez, J.C.M.: Super-resolution for hyperspectral and multispectral image fusion accounting for seasonal spectral variability. IEEE Trans. Image Process. 29, 116–127 (2020)
Lin, C.-H., Ma, F., Chi, C.-Y., Hsieh, C.-H.: A convex optimization-based coupled nonnegative matrix factorization algorithm for hyperspectral and multispectral data fusion. IEEE Trans. Geosci. Remote Sens. 56(3), 1652–1667 (2018)
Zhou, Y., Feng, L., Hou, C., Kung, S.: Hyperspectral and multispectral image fusion based on local low rank and coupled spectral unmixing. IEEE Trans. Geosci. Remote Sens. 55(10), 5997–6009 (2017)
Yokoya, N., Yairi, T., Iwasaki, A.: Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans. Geosci. Remote Sens. 50(2), 528–537 (2012)
Simões, M., Bioucas-Dias, J., Almeida, L.B., Chanussot, J.: A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Trans. Geosci. Remote Sens. 53(6), 3373–3388 (2015)
Lin, B., Tao, X., Mai, X., Dong, L., Jianhua, L.: Bayesian hyperspectral and multispectral image fusions via double matrix factorization. IEEE Trans. Geosci. Remote Sens. 55(10), 5666–5678 (2017)
Akhtar, N., Shafait, F., Mian, A.: Sparse spatio-spectral representation for hyperspectral image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 63–78. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10584-0_5
Dian, R., Li, S., Fang, L., Lu, T., Bioucas-Dias, J.M.: Nonlocal sparse tensor factorization for semiblind hyperspectral and multispectral image fusion. IEEE Trans. Cybern. 50, 4469–4480 (2019)
Zhou, Y., Rangarajan, A., Gader, P.D.: An integrated approach to registration and fusion of hyperspectral and multispectral images. IEEE Trans. Geosci. Remote Sens. 58(5), 3020–3033 (2020)
Qu, Y., Qi, H., Kwan, C.: Unsupervised and unregistered hyperspectral image super-resolution with mutual Dirichlet-Net. arXiv preprint arXiv:1904.12175 (2019)
Karoui, M.S., Deville, Y., Benhalouche, F.Z., Boukerch, I.: Hypersharpening by joint-criterion nonnegative matrix factorization. IEEE Trans. Geosci. Remote Sens. 55(3), 1660–1670 (2017)
Lanaras, C., Baltsavias, E., Schindler, K.: Hyperspectral super-resolution by coupled spectral unmixing. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3586–3594, December 2015
Lin, C.-J.: Projected gradient methods for nonnegative matrix factorization. Neural Comput. 19(10), 2756–2779 (2007)
Bertsekas, D.P.: On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Autom. Control 21(2), 174–184 (1976)
Nascimento, J.M.P., Dias, J.M.B.: Vertex component analysis: a fast algorithm to unmix hyperspectral data. IEEE Trans. Geosci. Remote Sens. 43(4), 898–910 (2005)
Acknowledgments
This work was supported in part by the National Key R&D Program of China under Grant 2017YFB0502900, in part by the National Natural Science Foundation of China under Grant 61772510, in part by the Innovation Capability Support Program of Shaanxi under Grant 2020TD-015.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, W., Lu, X. (2020). Unregistered Hyperspectral and Multispectral Image Fusion with Synchronous Nonnegative Matrix Factorization. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12305. Springer, Cham. https://doi.org/10.1007/978-3-030-60633-6_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-60633-6_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60632-9
Online ISBN: 978-3-030-60633-6
eBook Packages: Computer ScienceComputer Science (R0)