Skip to main content
SpringerLink
Log in

Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Restoration

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Hyperspectral imaging is beneficial in a diverse range of applications from diagnostic medicine, to agriculture, to surveillance to name a few. However, hyperspectral images often suffer from degradation such as noise and low resolution. In this paper, we propose an effective model for hyperspectral image (HSI) restoration, specifically image denoising and super-resolution. Our model considers three underlying characteristics of HSIs: sparsity across the spatial-spectral domain, high correlation across spectra, and non-local self-similarity over space. We first exploit high correlation across spectra and non-local self-similarity over space in the degraded HSI to learn an adaptive spatial-spectral dictionary. Then, we employ the local and non-local sparsity of the HSI under the learned spatial-spectral dictionary to design an HSI restoration model, which can be effectively solved by an iterative numerical algorithm with parameters that are adaptively adjusted for different clusters and different noise levels. In experiments on HSI denoising, we show that the proposed method outperforms many state-of-the-art methods under several comprehensive quantitative assessments. We also show that our method performs well on HSI super-resolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Due to space limitations, we chose 10 typical and complex scenes to test our method.

References

  • Aharon, M., Elad, M., & Bruckstein, A. (2006). K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Image Processing, 54(11), 4311–4322.

    Article  Google Scholar 

  • Akgun, T., Altunbasak, Y., & Mersereau, R. (2005). Super-resolution reconstruction of hyperspectral images. IEEE Transactions on Image Processing, 14(11), 1860–1875.

    Article  Google Scholar 

  • Atkinson, I., Kamalabadi, F. & Jones, D. (2003). Wavelet-based hyperspectral image estimation. In IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2, 743–745.

  • Banerjee, A., Burlina, P. & Broadwater, J. (2009). Hyperspectral video for illumination-invariant tracking. Evolution in Remote Sensing (WHISPERS). In Workshop on Hyperspectral Image and Signal Processing (pp. 1–4).

  • Bioucas-Dias, J., & Figueiredo, M. (2007). A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing, 16(12), 2992–3004.

    Article  MathSciNet  Google Scholar 

  • Borengasser, M., Hungate, W. S., & Watkins, R. (2007). Hyperspectral remote sensing: Principles and applications. Boca Raton: CRC Press.

    Google Scholar 

  • Bourennane, S., Fossati, C., & Cailly, A. (2010). Improvement of classification for hyperspectral images based on tensor modeling. IEEE Geoscience and Remote Sensing Letters, 7(4), 801–805.

    Article  Google Scholar 

  • Buades, A., Coll, B. & Morel, J. M. (2005). A non-local algorithm for image denoising. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2) (pp 60–65).

  • Buades, A., Coll, B., Morel, J. M., & Sbert, C. (2009). Self-similarity driven color demosaicking. IEEE Transactions on Image Processing, 18(6), 1192–1202.

    Article  MathSciNet  Google Scholar 

  • Buttingsrud, B. & Alsberg, B. (2006). Superresolution of hyperspectral images. Chemometrics and Intelligent Laboratory Systems (pp 62–68).

  • Candes, E., & Tao, T. (2006). Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Transactions on Information Theory, 52(12), 5406–5425.

    Article  MathSciNet  MATH  Google Scholar 

  • Castrodad, A., Xing, Z., Greer, J., Bosch, E., Carin, L., & Sapiro, G. (2011). Learning discriminative sparse representations for modeling, source separation, and mapping of hyperspectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 49(11), 4263–4281.

    Article  Google Scholar 

  • Chakrabarti, A. & Zickler, T. (2011). Statistics of real-world hyperspectral images. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp 193–200).

  • Chan, J., Ma, J., Kempeneers, P., & Canters, F. (2010). Superresolution enhancement of hyperspectral CHRIS/proba images with a thin-plate spline nonrigid transform model. IEEE Transaction on Geoscience and Remote Sensing, 48(6), 2569–2579.

    Article  Google Scholar 

  • Chen, C., Yeqing, L., Wei, L., Junzhou, H. (2014). Image fusion with local spectral consistency and dynamic gradient sparsity. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Chen, G., & Qian, S. E. (2011). Denoising of hyperspectral imagery using principal component analysis and wavelet shrinkage. IEEE Transactions on Geoscience and Remote Sensing, 49(3), 973–980.

    Article  Google Scholar 

  • Chen, S., Donoho, D., & Saunders, M. (1998). Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20(1), 33–61.

    Article  MathSciNet  MATH  Google Scholar 

  • Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.

    Article  MathSciNet  Google Scholar 

  • David, T., & Dicker, J. L. (2006). Differentiation of normal skin and melanoma using high resolution hyperspectral imaging. Cancer Biology and Therapy, 5(8), 1033–1038.

    Article  Google Scholar 

  • Dong, W., Li, X., Zhang, L. & Shi, G. (2011). Sparsity-based image denoising via dictionary learning and structural clustering. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp 457–464).

  • Dong, W., Zhang, L., Shi, G., & Li, X. (2013). Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing, 22(4), 1620–1630.

    Article  MathSciNet  Google Scholar 

  • Dong, W., Shi, G., Li, X., & Ma, Y. (2015). Image restoration via simultaneous sparse coding: Where structured sparsity meets gaussian scale mixture. International Journal of Computer Vision, 114(2), 217–232.

    Article  MathSciNet  Google Scholar 

  • Dong, W., Fu, F., Shi, G., Cao, X., Wu, J., Li, G., et al. (2016). Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Transactions on Image Processing, 25(5), 2337–2352.

    Article  MathSciNet  Google Scholar 

  • Donoho, D. L. (2004). For most large underdetermined systems of linear equations the minimal 1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 59, 797–829.

    Article  MathSciNet  MATH  Google Scholar 

  • Eismann, M., & Hardie, R. (2004). Application of the stochastic mixing model to hyperspectral resolution enhancement. IEEE Transactions on Geoscience and Remote Sensing, 42(9), 1924–1933.

    Article  Google Scholar 

  • Eismann, M., & Hardie, R. (2005). Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions. IEEE Transactions on Geoscience and Remote Sensing, 43(3), 455–465.

    Article  Google Scholar 

  • Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736–3745.

    Article  MathSciNet  Google Scholar 

  • Glasner, D., Bagon, S. & Irani, M. (2009). Super-resolution from a single image. In Proceedings of International Conference on Computer Vision (ICCV) (pp 349–356).

  • Guangyi, Chen, & Q, S. E. (2009). Denoising and dimensionality reduction of hyperspectral imagery using wavelet packets, neighbour shrinking and principal component analysis. International Journal of Remote Sensing, 30(18), 4889–4895.

    Article  Google Scholar 

  • Guo, X., Huang, X., Zhang, L., & Zhang, L. (2013). Hyperspectral image noise reduction based on rank-1 tensor decomposition. ISPRS Journal of Photogrammetry and Remote Sensing, 83, 50–63.

    Article  Google Scholar 

  • Gupta, N., & Ramella-Roman, J.C. (2008). Detection of blood oxygen level by noninvasive passive spectral imaging of skin. In Proceedings of SPIE.

  • Hou, B., Zhang, X., Ye, Q., & Zheng, Y. (2013). A novel method for hyperspectral image classification based on laplacian eigenmap pixels distribution-flow. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(3), 1602–1618.

    Article  Google Scholar 

  • Ingrid Daubechies, M. D. (2004). An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics, 57(11), 1413–1457.

    Article  MathSciNet  MATH  Google Scholar 

  • Kang, X., Li, S., & Benediktsson, J. (2014). Feature extraction of hyperspectral images with image fusion and recursive filtering. IEEE Transactions on Geoscience and Remote Sensing, 52(6), 3742–3752.

    Article  Google Scholar 

  • Karami, A., Yazdi, M., & Zolghadre Asli, A. (2011). Noise reduction of hyperspectral images using kernel non-negative tucker decomposition. IEEE Journal of Selected Topics in Signal Processing, 5(3), 487–493.

    Article  Google Scholar 

  • Lam, A., Sato, I., Sato, Y. (2012). Denoising hyperspectral images using spectral domain statistics. In Proceedings of International Conference on Pattern Recognition (ICPR) (pp. 477–480).

  • Letexier, D., & Bourennane, S. (2008). Noise removal from hyperspectral images by multidimensional filtering. IEEE Transactions on Geoscience and Remote Sensing, 46(7), 2061–2069.

    Article  Google Scholar 

  • Lu, G., & Fei, B. (2014). Medical hyperspectral imaging: A review. Journal of Biomedical Optics, 19(1), 010901.

    Article  Google Scholar 

  • Ma, C., Cao, X., Tong, X., Dai, Q., & Lin, S. (2013). Acquisition of high spatial and spectral resolution video with a hybrid camera system. International Journal of Computer Vision, 110, 141–155.

    Article  Google Scholar 

  • Maggioni, M., Katkovnik, V., Egiazarian, K., & Foi, A. (2013). Nonlocal transform-domain filter for volumetric data denoising and reconstruction. IEEE Transactions on Image Processing, 22(1), 119–133.

    Article  MathSciNet  Google Scholar 

  • Mairal, J, Bach, F., Ponce, J. & Sapiro, G. (2009). Online dictionary learning for sparse coding. In Proceedings of International Conference on Machine Learning (ICML) (pp 689–696).

  • Mairal, J., Bach, F., Ponce, J., Sapiro, G. & Zisserman. A. (2009). Non-local sparse models for image restoration. In Proceedings of International Conference on Computer Vision (ICCV) (pp 2272–2279).

  • Manjn, J. V., Coup, P., Mart-Bonmat, L., Collins, D. L., & Robles, M. (2010). Adaptive non-local means denoising of MR images with spatially varying noise levels. Journal of Magnetic Resonance Imaging, 31(1), 192–203.

    Article  Google Scholar 

  • Melgani, F., & Bruzzone, L. (2004). Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing, 42(8), 1778–1790.

    Article  Google Scholar 

  • Monno, Y., Tanaka, M., Okutomi, M. (2013). Direct spatio-spectral datacube reconstruction from raw data using a spatially adaptive spatio-spectral basis. In Proceedings of SPIE 8660, Digital Photography IX vol 8660 (pp. 866,003–866,003–8)

  • Murakami, Y., Fukura, K., Yamaguchi, M., & Ohyama, N. (2008). Color reproduction from low-SNR multispectral images using spatio-spectral Wiener estimation. Optics Express, 16(6), 4106.

    Article  Google Scholar 

  • Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583), 607–609.

    Article  Google Scholar 

  • Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Research, 37(23), 3311–3325.

    Article  Google Scholar 

  • Othman, H., & Qian, S. E. (2006). Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage. IEEE Transactions on Geoscience and Remote Sensing, 44(2), 397–408.

    Article  Google Scholar 

  • Parmar, M., Lansel, S. & Wandell, B. (2008). Spatio-spectral reconstruction of the multispectral datacube using sparse recovery. In Proceedings of IEEE International Conference on Image Processing (ICIP) (pp 473–476).

  • Peng, Y., Meng, D., Xu, Z., Gao, C., Yang, Y. & Zhang, B. (2014). Decomposable nonlocal tensor dictionary learning for multispectral image denoising. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Qian, Y., & Ye, M. (2013). Hyperspectral imagery restoration using nonlocal spectral-spatial structured sparse representation with noise estimation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(2), 499–515.

    Article  Google Scholar 

  • Qian, Y., Shen, Y., Ye, M. & Wang, Q. (2012). 3-D Nonlocal means filter with noise estimation for hyperspectral imagery denoising. In IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (pp. 1345–1348).

  • Renard, N., & Bourennane, S. (2008). Improvement of target detection methods by multiway filtering. IEEE Transactions on Geoscience and Remote Sensing, 46(8), 2407–2417.

    Article  Google Scholar 

  • Renard, N., Bourennane, S., & Blanc-Talon, J. (2008). Denoising and dimensionality reduction using multilinear tools for hyperspectral images. IEEE Geoscience and Remote Sensing Letters, 5(2), 138–142.

    Article  Google Scholar 

  • Rowe, R., Nixon, K. & Corcoran, S. (2005). Multispectral fingerprint biometrics. In Proceedings of IEEE Information Assurance Workshop (IAW) (pp 14–20).

  • Shimano, M., Okabe, T., Sato, I. & Sato, Y. (2011). Video temporal super-resolution based on self-similarity. In Proceedings of Asian Conference on Computer Vision (ACCV) (pp 93–106).

  • Stamatas, G.N., Balas, C.J., Kollias, N. (2003). Hyperspectral image acquisition and analysis of skin. In Proceedings of SPIE (pp. 77–82).

  • Teke, M., Deveci, H., Haliloglu, O., Gurbuz, S. & Sakarya, U. (2013). A short survey of hyperspectral remote sensing applications in agriculture. In International Conference on Recent Advances in Space Technologies (RAST) (pp 171–176).

  • Vrhel, M. J., Gershon, R., & Iwan, L. S. (1994). Measurement and analysis of object reflectance spectra. Color Research and Application, 19(1), 4–9.

    Google Scholar 

  • Wang, Y. & Niu, R. (2009). Hyperspectral urban remote sensing image smoothing and enhancement using forward-and-backward diffusion. In Joint Urban Remote Sensing Event (pp. 1–5).

  • Wang, Y., Niu, R., & Yu, X. (2010). Anisotropic diffusion for hyperspectral imagery enhancement. IEEE Sensors Journal, 10(3), 469–477.

    Article  Google Scholar 

  • Wang, Z., Bovik, A., Sheikh, H., & Simoncelli, E. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transaction on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Yang, C. Y., Huang, J. B. & Yang, M. H. (2011). Exploiting self-similarities for single frame super-resolution. In Proceedings of Asian Conference on Computer Vision (ACCV) (pp 497–510).

  • Yang, J., Wright, J., Huang, T., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 19(11), 2861–2873.

    Article  MathSciNet  Google Scholar 

  • Yasuma, F., Mitsunaga, T., Iso, D. & Nayar, S. (2008). Generalized assorted pixel camera: Post-capture control of resolution, dynamic range and spectrum. Tech. rep.

  • Yuan, Q., Zhang, L., & Shen, H. (2012). Hyperspectral image denoising employing a spectral-spatial adaptive total variation model. IEEE Transactions on Geoscience and Remote Sensing, 50(10), 3660–3677.

    Article  Google Scholar 

  • Zhang, H., Zhang, L., & Shen, H. (2012). A super-resolution reconstruction algorithm for hyperspectral images. Signal Processing, 92(9), 2082–2096.

    Article  Google Scholar 

  • Zhang, H., He, W., Zhang, L., Shen, H., & Yuan, Q. (2014). Hyperspectral image restoration using low-rank matrix recovery. IEEE Transactions on Geoscience and Remote Sensing, 52(8), 4729–4743.

    Article  Google Scholar 

  • Zhang, L., Zhang, D., Mou, X., & Zhang, D. (2011). FSIM: A feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 20(8), 2378–2386.

    Article  MathSciNet  Google Scholar 

  • Zhao, Q., Meng, D., Kong, X., Xie, Q., Cao, W., Wang, Y. & Xu, Z. (2015). A novel sparsity measure for tensor recovery. In Proceedings of International Conference on Computer Vision (ICCV) (pp 271–279).

  • Zhao, Y., Yang, J., Zhang, Q., Song, L., Cheng, Y., & Pan, Q. (2011). Hyperspectral imagery super-resolution by sparse representation and spectral regularization. EURASIP Journal on Advances in Signal Processing, 1, 1–10.

    Google Scholar 

  • Zhong, P., & Wang, R. (2013). Multiple-spectral-band CRFs for denoising junk bands of hyperspectral imagery. IEEE Transactions on Geoscience and Remote Sensing, 51(4), 2260–2275.

    Article  Google Scholar 

  • Zibulevsky, M., & Elad, M. (2010). L1–l2 optimization in signal and image processing. IEEE Transactions on Signal Processing Magazine, 27(3), 76–88.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo, 153-8505, Japan

    Ying Fu & Yoichi Sato

  2. Graduate School of Science and Engineering, Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama, 338-8570, Japan

    Antony Lam

  3. National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda, Tokyo, 101-8430, Japan

    Imari Sato

Authors
  1. Ying Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antony Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Imari Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yoichi Sato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Fu.

Additional information

Communicated by Hiroshi Ishikawa, Takeshi Masuda, Yasuyo Kita and Katsushi Ikeuchi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, Y., Lam, A., Sato, I. et al. Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Restoration. Int J Comput Vis 122, 228–245 (2017). https://doi.org/10.1007/s11263-016-0921-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-016-0921-6

Keywords

Search

Navigation