Skip to main content
SpringerLink
Log in

Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In image processing, sparse coding has been known to be relevant to both variational and Bayesian approaches. The regularization parameter in variational image restoration is intrinsically connected with the shape parameter of sparse coefficients’ distribution in Bayesian methods. How to set those parameters in a principled yet spatially adaptive fashion turns out to be a challenging problem especially for the class of nonlocal image models. In this work, we propose a structured sparse coding framework to address this issue—more specifically, a nonlocal extension of Gaussian scale mixture (GSM) model is developed using simultaneous sparse coding (SSC) and its applications into image restoration are explored. It is shown that the variances of sparse coefficients (the field of scalar multipliers of Gaussians)—if treated as a latent variable—can be jointly estimated along with the unknown sparse coefficients via the method of alternating optimization. When applied to image restoration, our experimental results have shown that the proposed SSC–GSM technique can both preserve the sharpness of edges and suppress undesirable artifacts. Thanks to its capability of achieving a better spatial adaptation, SSC–GSM based image restoration often delivers reconstructed images with higher subjective/objective qualities than other competing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Throughout this paper, we will use subscript/superscript to denote column/row vectors of a matrix respectively.

References

  • Aharon, M., Elad, M., & Bruckstein, A. (2012). The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representations. IEEE Transactions on Signal Processing, 54(11), 4311–4322.

    Article  Google Scholar 

  • Andrews, D. F., & Mallows, C. L. (1974). Scale mixtures of normal distributions. Journal of the Royal Statistical Society. Series B (Methodological), 36(1), 99–102.

    MathSciNet  MATH  Google Scholar 

  • Beck, A., & Teboulle, M. (2009). Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Transactions on Image Processing, 18(11), 2419–2434.

    Article  MathSciNet  Google Scholar 

  • Box, G. E., & Tiao, G. C. (2011). Bayesian inference in statistical analysis (Vol. 40). New York: Wiley.

    Google Scholar 

  • Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.

    Article  Google Scholar 

  • Buades, A., Coll, B., & Morel, J.-M. (2005). A non-local algorithm for image denoising. CVPR, 2, 60–65.

    Google Scholar 

  • Candes, E., Wakin, M., & Boyd, S. (2008). Enhancing sparsity by reweighted \(l_1\) minimization. Journal of Fourier Analysis and Applications, 14(5), 877–905.

    Article  MathSciNet  MATH  Google Scholar 

  • Carlson, C., Adelson, E., & Anderson, C. (Jun 1985). System for coring an image-representing signal. US Patent 4,523,230.

  • Chang, S. G., Yu, B., & Vetterli, M. (2000). Adaptive wavelet thresholding for image denoising and compression. IEEE Transactions on Image Processing, 9(9), 1532–1546.

    Article  MathSciNet  MATH  Google Scholar 

  • Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095.

    Article  MathSciNet  Google Scholar 

  • Danielyan, A., Katkovnik, V., & Egiazarian, K. (2012). Bm3d frames and variational image deblurring. IEEE Transactions on Image Processing, 21(4), 1715–1728.

    Article  MathSciNet  Google Scholar 

  • Daubechies, I. (1988). Orthonormal bases of compactly supported bases. Communications On Pure and Applied Mathematics, 41, 909–996.

    Article  MathSciNet  MATH  Google Scholar 

  • Do, M. N., & Vetterli, M. (2005). The contourlet transform: An efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12), 2091–2106.

    Article  MathSciNet  Google Scholar 

  • Dong, W., Li, X., Zhang, L., & Shi, G. (2011). Sparsity-based image denoising via dictionary learning and structural clustering. In: IEEE Conference on Computer Vision and Pattern Recognition.

  • Dong, W., Shi, G., & Li, X. (2013a). Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Transactions on Image Processing, 22(2), 700–711.

  • Dong, W., Zhang, L., Shi, G., & Li, X. (2013b). Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing, 22(4), 1620–1630.

  • Donoho, D., & Johnstone, I. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81, 425–455.

    Article  MathSciNet  MATH  Google Scholar 

  • Elad, M., & Aharon, M. (2012). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 21(9), 3850–3864.

    Article  MathSciNet  Google Scholar 

  • Garrigues, P., & Olshausen, B. A. (2010). Group sparse coding with a laplacian scale mixture prior. In: Advances in neural information processing systems, (pp. 676–684).

  • Gasso, G., Rakotomamonjy, A., & Canu, S. (2009). Recovering sparse signals with a certain family of nonconvex penalties and DC programming. IEEE Transactions on Signal Processing, 57(12), 4686–4698.

    Article  MathSciNet  Google Scholar 

  • Hocking, R. R. (1976). A biometrics invited paper: The analysis and selection of variables in linear regression. Biometrics, 32(1), 1–49.

  • Ji, S., Xue, Y., & Carin, L. (2008). Bayesian compressive sensing. IEEE Transactions on Signal Processing, 56(6), 2346–2356.

    Article  MathSciNet  Google Scholar 

  • Katkovnik, V., Foi, A., Egiazarian, K., & Astola, J. (2010). From local kernel to nonlocal multiple-model image denoising. International Journal of Computer Vision, 86(1), 1–32.

    Article  MathSciNet  Google Scholar 

  • Lyu, S., & Simoncelli, E. (2009). Modeling multiscale subbands of photographic images with fields of gaussian scale mixtures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4), 693–706.

    Article  Google Scholar 

  • Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009a). Online dictionary learning for sparse coding. In: 2009 IEEE 26th International Conference on Machine Learning, (pp. 689–696).

  • Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009b). Non-local sparse models for image restoration. In: 2009 IEEE 12th International Conference on Computer Vision, (pp. 2272–2279).

  • Mairal, J., Sapiro, G., & Elad, M. (2008). Learning multiscale sparse representation for image and video restoration. SIAM Multiscale Modeling and Simulation, 7(1), 214–241.

    Article  MathSciNet  MATH  Google Scholar 

  • Mallat, S. (1989). Multiresolution approximations and wavelet orthonormal bases of \(l^{2}({\varvec {r}})\). Transactions of the American Mathematical Society, 315, 69–87.

    MathSciNet  Google Scholar 

  • Marquina, A., & Osher, S. J. (2008). Image super-resolution by tv-regularization and bregman iteration. Journal of Scientific Computing, 37(3), 367–382.

    Article  MathSciNet  MATH  Google Scholar 

  • Mihcak, I. K. M.K., & Ramchandran, K. (1999). Local statistical modeling of wavelet image coefficients and its application to denoising. In: IEEE International Conference on Acoust. Speech Signal Processing, (pp. 3253–3256).

  • Polson, N. G., & Scott, J. G. (2010). Shrink globally, act locally: sparse bayesian regularization and prediction. Bayesian Statistics, 9, 501–538.

  • Portilla, J., Strela, V., Wainwright, M., & Simoncelli, E. (2003). Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12, 1338–1351.

    Article  MathSciNet  MATH  Google Scholar 

  • Ramirez, I., & Sapiro, G. (2012). Universal regularizers for robust sparse coding and modeling. IEEE Transactions on Image Processing, 21(9), 3850–3864.

    Article  MathSciNet  Google Scholar 

  • Said, A., & Pearlman, W. A. (1996). A new fast and efficient image codec based on set partitioning in hierarchical trees. IEEE Transactions on Circuits and Systems for Video Technology, 6, 243–250.

    Article  Google Scholar 

  • Shapiro, J. M. (1993). Embedded image coding using zerotrees of wavelet coefficients. EEE Transactions on Acoustics, Speech and Signal Processing, 41(12), 3445–3462.

    Article  MATH  Google Scholar 

  • Taubman, D., & Marcellin, M. (2001). JPEG2000: Image compression fundamentals, standards, and practice. Norwell: Kluwer.

  • Tipping, M. (2001). Sparse bayesian learning and the relevance vector machine. The Journal of Machine Learning Research, 1, 211–244.

  • Vetterli, M. (1986). Filter banks allowing perfect reconstruction. Signal Processing, 10(3), 219–244.

    Article  MathSciNet  Google Scholar 

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Wipf, D. P., & Rao, B. D. (2004). Sparse bayesian learning for basis selection. IEEE Transactions on Signal Processing, 52(8), 2153–2164.

    Article  MathSciNet  Google Scholar 

  • Wipf, D. P., & Rao, B. D. (2007). An empirical Bayesian strategy for solving the simultaneous sparse approximation problem. IEEE Transactions on Signal Processing, 55(7), 3704–3716.

    Article  MathSciNet  Google Scholar 

  • Wipf, D. P., Rao, B. D., & Nagarajan, S. (2011). Latent variable bayesian models for promoting sparsity. IEEE Transactions on Information Theory, 57(9), 6236–6255.

    Article  MathSciNet  Google Scholar 

  • Xu, J., & Osher, S. (2007). Iterative regularization and nonlinear inverse scale space applied to wavelet-based denoising. IEEE Transactions on Image Processing, 16(2), 534–544.

    Article  MathSciNet  Google Scholar 

  • Yang, J., Wright, J., Huang, T., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 19(11), 2861–2873.

    Article  MathSciNet  Google Scholar 

  • Yu, G., Sapiro, G., & Mallat, S. (2012). Solving inverse problems with piecewise linear estimators: from gaussian mixture models to structured sparsity. IEEE Transactions on Image Processing, 21(5), 2481–2499.

    Article  MathSciNet  Google Scholar 

  • Zhou, M., Chen, H., Paisley, J., Ren, L., Sapiro, G., & Carin, L. (2009). Non-parametric bayesian dictionary learning for sparse image representation. In: Advances in neural information processing systems, (pp. 2295–2303).

  • Zhou, M., Chen, H., Paisley, J., Ren, L., Li, L., Xing, Z., et al. (2012). Nonparametric bayesian dictionary learning for analysis of noisy and incomplete images. IEEE Transactions on Image Processing, 21(1), 130–144.

    Article  MathSciNet  MATH  Google Scholar 

  • Zoran, D., & Weiss, Y. (2011). From learning models of natural image patches to whole image restoration. In: Proceedings of ICCV.

Download references

Acknowledgments

The authors would like to thank Zhouchen Lin of Peking University for helpful discussion. They would also like to thank the three anonymous reviewers for their valuable comments and constructive suggestions that have much improved the presentation of this paper. This work was supported in part by the Major State Basic Research Development Program of China (973 Program) under Grant 2013CB329402, in part by the Natural Science Foundation (NSF) of China under Grant 61471281, Grant 61227004 and Grant 61390512, in part by the Program for New Scientific and Technological Star of Shaanxi Province under Grant 2014KJXX-46, in part by the Fundamental Research Funds of the Central Universities of China under Grant BDY081424 and Grant K5051399020, and in part by NSF under Award ECCS-0968730.

Author information

Authors and Affiliations

  1. School of Electronic Engineering, Xidian University, Xi’an, 710071, China

    Weisheng Dong & Guangming Shi

  2. School of Information Science and Technology, ShanghaiTech University, Shanghai, China

    Yi Ma

  3. Lane Department of CSEE, West Virginia University, Morgantown, WV, 26506-6109, USA

    Xin Li

Authors
  1. Weisheng Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangming Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weisheng Dong.

Additional information

Communicated by Julien Mairal, Francis Bach, Michael Elad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, W., Shi, G., Ma, Y. et al. Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture. Int J Comput Vis 114, 217–232 (2015). https://doi.org/10.1007/s11263-015-0808-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-015-0808-y

Keywords

Search

Navigation