Skip to main content
SpringerLink
Log in
Book cover

International Forum on Digital TV and Wireless Multimedia Communications

IFTC 2019: Digital TV and Wireless Multimedia Communication pp 49–61Cite as

Hyperspectral Image Super-Resolution Using Multi-scale Feature Pyramid Network

  • Conference paper
  • First Online:

  • 762 Accesses

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1181))

Abstract

Hyperspectral (HS) images are captured with rich spectral information, which have been proved to be useful in many real-world applications, such as earth observation. Due to the limitations of HS cameras, it is difficult to obtain HS images with high-resolution (HR). Recent advances in deep learning (DL) for single image super-resolution (SISR) task provide a powerful tool for restoring high-frequency details from low-resolution (LR) input image. Inspired by this progress, in this paper, we present a novel DL-based model for single HS image super-resolution in which a feature pyramid block is designed to extract multi-scale features of the input HS image. Our method does not need auxiliary inputs which further extends the application scenes. Experiment results show that our method outperforms state-of-the-arts on both objective quality indices and subjective visual results.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bioucas-Dias, J.M., et al.: Hyperspectral unmixing overview: geometrical, statistical, and sparse regression-based approaches. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 5, 354–379 (2012)

    Article  Google Scholar 

  2. Haut, J.M., Paoletti, M.E., Plaza, J., Li, J.Y., Plaza, A.J.: Active learning with convolutional neural networks for hyperspectral image classification using a new bayesian approach. IEEE Trans. Geosci. Remote Sens. 56, 6440–6461 (2018)

    Article  Google Scholar 

  3. Camps-Valls, G., Bruzzone, L.: Kernel-based methods for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 43, 1351–1362 (2005)

    Article  Google Scholar 

  4. Nguyen, H.V., Banerjee, A., Chellappa, R.: Tracking via object reflectance using a hyperspectral video camera. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 44–51 (2010)

    Google Scholar 

  5. Huete, A.R., Miura, T., Gao, X.: Land cover conversion and degradation analyses through coupled soil-plant biophysical parameters derived from hyperspectral EO-1 hyperion. IEEE Trans. Geosci. Remote Sens. 41, 1268–1276 (2003)

    Article  Google Scholar 

  6. Roberts, D.A., Dennison, P.E., Gardner, M.E., Hetzel, Y., Ustin, S.L., Lee, C.T.: Evaluation of the potential of hyperion for fire danger assessment by comparison to the airborne visible/infrared imaging spectrometer. IEEE Trans. Geosci. Remote Sens. 41, 1297–1310 (2003)

    Article  Google Scholar 

  7. Simões, M., Bioucas-Dias, J.M., Almeida, L.B., Chanussot, J.: A convex formulation for hyperspectral image superresolution via subspace-based regularization. IEEE Trans. Geosci. Remote Sens. 53, 3373–3388 (2015)

    Article  Google Scholar 

  8. Wei, Q., Bioucas-Dias, J.M., Dobigeon, N., Tourneret, J.Y.: Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53, 3658–3668 (2014)

    Article  Google Scholar 

  9. Yokoya, N., Yairi, T., Iwasaki, A.: Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans. Geosci. Remote Sens. 50, 528–537 (2012)

    Article  Google Scholar 

  10. Zhang, K., Wang, M., Yang, S.Y., Jiao, L.: Spatialspectral-graph-regularized low-rank tensor decomposition for multispectral and hyperspectral image fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 11, 1030–1040 (2018)

    Article  Google Scholar 

  11. Qu, Y., Qi, H., Kwan, C.: Unsupervised sparse Dirichlet-Net for hyperspectral image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2511–2520 (2018)

    Google Scholar 

  12. Xie, Q., Zhou, M., Zhao, Q., Meng, D., Zuo, W., Xu, Z.: Multispectral and hyperspectral image fusion by MS/HS fusion net. ArXiv abs/1901.03281 (2019)

    Google Scholar 

  13. Loncan, L., et al.: Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3, 27–46 (2015)

    Article  Google Scholar 

  14. Akgun, T., Altunbasak, Y., Mersereau, R.M.: Super-resolution reconstruction of hyperspectral images. IEEE Trans. Image Process. 14, 1860–1875 (2005)

    Article  Google Scholar 

  15. Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., Du, Q.: Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sens. 9, 1139 (2017)

    Article  Google Scholar 

  16. Yuan, Y., Zheng, X., Lu, X.: Hyperspectral image superresolution by transfer learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(5), 1963–1974 (2017)

    Article  Google Scholar 

  17. Li, Y., Zhang, L., Ding, C., Wei, W., Zhang, Y.: Single hyperspectral image super-resolution with grouped deep recursive residual network. In: 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), pp. 1–4 (2018)

    Google Scholar 

  18. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38, 295–307 (2014)

    Article  Google Scholar 

  19. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2015)

    Google Scholar 

  20. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1132–1140 (2017)

    Google Scholar 

  21. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5835–5843 (2017)

    Google Scholar 

  22. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2472–2481 (2018)

    Google Scholar 

  23. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. ArXiv abs/1807.02758 (2018)

    Google Scholar 

  24. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2017)

    Google Scholar 

  25. Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1664–1673 (2018)

    Google Scholar 

  26. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  27. Lin, T.Y., Dollár, P., Girshick, R.B., He, K., Hariharan, B., Belongie, S.J.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2016)

    Google Scholar 

  28. He, K., Gkioxari, G., Dollar, P., Girshick, R.B.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. (2018)

    Google Scholar 

  29. Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1874–1883 (2016)

    Google Scholar 

  30. Yokoya, N., Iwasaki, A.: Airborne hyperspectral data over Chikusei. Technical report SAL-2016-05-27, Space Application Laboratory, University of Tokyo, Japan (May 2016)

    Google Scholar 

  31. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)

    Google Scholar 

  32. Sidorov, O., Hardeberg, J.Y.: Deep hyperspectral prior: denoising, inpainting, super-resolution. ArXiv abs/1902.00301 (2019)

    Google Scholar 

  33. Ulyanov, D., Vedaldi, A., Lempitsky, V.S.: Deep image prior. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9446–9454 (2017)

    Google Scholar 

  34. Wald, L.: Data Fusion, Definitions and Architectures - Fusion of Images of Different Spatial Resolutions. Les Presses de l’Ecole des Mines, Paris (2002)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Science Foundation under Grant Nos. 61971165, 61672193, and 61922027, and is also supported by the Fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China

    He Sun, Zhiwei Zhong, Deming Zhai, Xianming Liu & Junjun Jiang

Authors
  1. He Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiwei Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deming Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xianming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junjun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junjun Jiang .

Editor information

Editors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, China

    Guangtao Zhai

  2. Shanghai Jiao Tong University, Shanghai, China

    Jun Zhou

  3. Shanghai Jiao Tong University, Shanghai, China

    Hua Yang

  4. Shanghai University, Shanghai, China

    Ping An

  5. Shanghai Jiao Tong University, Shanghai, China

    Xiaokang Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sun, H., Zhong, Z., Zhai, D., Liu, X., Jiang, J. (2020). Hyperspectral Image Super-Resolution Using Multi-scale Feature Pyramid Network. In: Zhai, G., Zhou, J., Yang, H., An, P., Yang, X. (eds) Digital TV and Wireless Multimedia Communication. IFTC 2019. Communications in Computer and Information Science, vol 1181. Springer, Singapore. https://doi.org/10.1007/978-981-15-3341-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-3341-9_5

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-3340-2

  • Online ISBN: 978-981-15-3341-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation