Elsevier

Neurocomputing

Volume 106, 15 April 2013, Pages 12-20
Neurocomputing

Sparse coding for image denoising using spike and slab prior

https://doi.org/10.1016/j.neucom.2012.09.014Get rights and content

Abstract

Sparse coding is a challenging and promising theme in image denoising. Its main goal is to learn a sparse representation from an over-complete dictionary. How to obtain a better sparse representation from the dictionary is important for the denoising process. In this paper, starting from the classic image denoising problem, a Bayesian-based sparse coding algorithm is proposed, which learns sparse representation with the spike and slab prior. Using the spike and slab prior, the proposed algorithm can achieve accurate prediction performance and effectively enforce sparsity. Experimental results on image denoising have demonstrated that the proposed algorithm can provide better representation and obtain excellent denoising performance.

Introduction

Image denoising is one of the most important areas of research in image processing [1], [2]. It has been a well-studied problem in the past several decades [3], [4]. The classic image denoising problem is formulated as [8]yi=xi+ϵi,where xi is the original image patch intensity written in a vectorized form, yi is the vectorized noisy image patch and ϵi is the vectorized noise patch. Denoising is such a typical inverse problem of calculating the patch intensity xi.

Several methods for solving the denoising problem can be presented in recent years. Total variation denoising method was firstly introduced by Rudin et al. in [29]. Although TV method can obtain considerable denoising performance, it generate the staircase effect. Portilla et al. performs denoising in the transform domain [30]. In non-local means-based method, Buades et al. [6] provided a simple patch-based method by exploiting the repeating structures of a given image, and suppressed the noise using a weighted averaging of pixels. An improved method was proposed by Kervrann and Boulanger [15], which used an iterative scheme to estimate the weights and the support region. Dabov et al. [10] proposed another denoising method named BM3D which utilizes a related idea of matching similar patches in noisy image. While it was unlike the previous two methods, each patch is performed to denoise in the transform domain [24]. In regression-based method, Takeda et al. proposed the iterative method of Steering Kernel Regression (SKR) [20]. The proposed method in [20] exploited the similarity of image pixels to calculate the noise-free pixel intensity and performed quite well. Image denoising problem was regarded as a regression problem, which is then solved using support vector regression (SVR) [16], [25]. Hou and Koh [13] proposed an robust regression method, which is stable and reliable for the Guassian noise and impulsive noise. In sparse representation-based method, Elad and Aharon [11] used the sparse coding (K-SVD) [5] for denoising problem and achieved excellent denoising results [24]. Each noisy image patch is handled by coding as a linear combination of only a few basis elements from an over-complete dictionary. In [9], Chatterjee and Milanfar proposed a locally learned dictionaries (K-LLD) method, which can be locally adaptive to denoise the noisy image patch in each cluster. Via setting a threshold value, each denoised patch was reconstructed by selecting the top m most informative principal components from local dictionaries learned by principal component analysis (PCA) [9].

Since the sparse coding based method can preserve the image texture information and filter the noise information, it can be used for image denoising. In image restoration, how to obtain a proper sparse representation from an over-complete dictionary is very important [19]. Hence, it is necessary to design a good sparse coding method to obtain properly sparse representations for image denoising. However, the traditional sparse coding based methods fail to consider the accurate prediction performance of the sparse representation. It has been demonstrated that accurate prediction of the representation is of prime importance for linear regression model, which can improve the learning performance [21]. Zou et al. proposed the elastic net method, which is regarded as a generalization of the lasso [26]. The elastic net method provides good prediction accuracy and a grouping effect, which can be used in classification problems [27]. However, the elastic net method penalizes true large coefficients more, which may cause biased estimation and produce over-penalization [28]. It is reasonable to believe that a good sparse coding method should consider the following aspects: the representation should have accurate prediction and effective sparsity performances simultaneously. It is known that the sparse representation with the spike and slab prior in the linear regression model can provide accurate prediction [12], [14]. Based on Bayesian framework, a novel sparse coding method is proposed, which incorporates the spike and slab prior for the representation.

In the Bayesian framework, prediction of the representation is to estimate the marginal posterior probability of a covariate in the linear regression model based on the observed data and prior information. Although the L1 norm penalty has good performance in sparse representation, it may cause biased estimation since it penalizes true large coefficients more, and thus may produce over-penalization and poor of prediction in regression model [28]. Sparse representation learning can generate the ability to address the image denoising problem as a direct sparse decomposition technique over redundant dictionaries. Moreover the accuracy prediction of the representations means that the learning processing is implemented by assuming a regression model can reduce overfitting and enforce appropriate priors. Hence, the accuracy prediction of the representations is of prime importance to obtain properly sparse representations for image denoising. Using spike and slab prior, the proposed method can provide accurate prediction and effectively enforce sparsity. The main contributions of this paper are listed as follows:

  • (1)

    Based on a Bayesian framework, the spike and slab prior is introduced to sparse coding. The proposed method inherits the advantage of Bayesian framework which can better select variables by appropriate priors.

  • (2)

    The proposed method can provide accurate predictions and effective sparsity by exploiting the spike and slab prior. Thus, the proposed method has the powerful discriminate ability in image denoising.

The rest of this paper is organized as follows. Section 2 introduces the problem formulation. Section 3 proposes a Bayesian-based sparse coding method using spike and slab priors and uses the proposed method for image denoising. In Section 4, experimental results are shown to demonstrate the effectiveness of the proposed method. Section 5 concludes this paper.

Section snippets

Problem formulation

In this section, the notations are presented in what follows. Let Y={y1,y2,,yn}Rm×n be a noisy image patch matrix. Let X={x1,x2,,xn}Rm×n be the denoised (and unknown) version of Y. Let D={d1,d2,,dK}Rm×K be an over-complete dictionary (K>m), where each di represents a basis vector in the dictionary. Assume that the dictionary D is known, each denoised patch xi can be approximatively represented as a sparse linear combination with respect to D. That is, the patch xi can be written as xi=Dsi

Sparse coding for image denoising

In this section, a novel sparse coding is proposed using the spike and slab prior under a Bayesian framework.

Experimental results

In this section, the proposed method performs in various experiments, with images corrupted with simulated noise, which is the Guassian additive noise. The parameters in the proposed method need to be explained and tuned to obtain the proper results in terms of PSNR. The size of the image patch is 8×8, which is commonplace in the literature of image denoising [11].

In our work, there are four important parameters to be explained. The first one is the size of the dictionary K, where the test

Conclusion

This paper proposes a sparse coding method using the spike and slab prior based on a Bayesian framework. The proposed method based on this Bayesian framework can provide accurate prediction and effective sparsity. The sparse representations obtained by the proposed method are applied into image denoising. Experimental results demonstrates that the proposed method can effectively deal with image denoising process and obtain excellent performance.

Acknowledgments

This work is supported by the National Basic Research Program of China (973 Program) (Grant No. 2011CB707104), by the National Natural Science Foundation of China (Grant Nos. 61100079, 61172142, and 61172143), and by the Postdoctoral Science Foundation of China (Grant No. Y11I971400).

Xiaoqiang Lu received the Ph.D. degree from Dalian University of Technology, Dalian, China. He is a postdoctoral researcher with the Center for Optical Imagery Analysis and Learning, State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His research interests include pattern recognition, machine learning, hyperspectral image analysis, cellular automata, and medical imaging.

References (30)

  • P. Chatterjee et al.

    Is denoising dead?

    IEEE Trans. Image Process.

    (2005)
  • P. Chatterjee et al.

    Clustering-based denoising with locally learned dictionaries

    IEEE Trans. Image Process.

    (2009)
  • K. Dabov et al.

    Image denoising by sparse 3-D transform-domain collaborative filtering

    IEEE Trans. Image Process.

    (2007)
  • M. Elad et al.

    Image denoising via sparse and redundant representations over learned dictionaries

    IEEE Trans. Image Processing

    (2006)
  • J. Hernández-Lobato, Balancing Flexibility and Robustness in Machine Learning: Semi-parametric Methods and Sparse...
  • Cited by (40)

    • A least angle regression assessment algorithm based on joint dictionary for visible and near-infrared spectrum denoising

      2021, Optik
      Citation Excerpt :

      However, most of the above methods are subjective, empirical and suboptimal, without exploring the prior knowledge of spectrum. Sparse representation has been used in many fields, such as feature selection [12], image compression [13], image denoising [14,15], and baseline correction [16,17]. Based on an over-completed dictionary [18] that contains prototype signal-atoms, sparse representation uses a minimum number of dictionary atoms to approximate the signal.

    • Adaptive algorithm for sparse signal recovery

      2019, Digital Signal Processing: A Review Journal
      Citation Excerpt :

      In Bayesian sparse signal recovery, setting priors for the signal has played a key role in promoting sparsity and improving performance. Examples of priors are spike and slab [27,24,2,31,30,39], Bernoulli–Gaussian [20], Bernoulli-exponential [12], Laplacian [3], and generalized Pareto [8]. Chaari et al. [11] explored a fully Bayesian sparse signal recovery using Bernoulli–Laplacian priors and obtained sparser solutions in comparison to the Bernoulli–Gaussian prior setting.

    • Stacked convolutional auto-encoders for single space target image blind deconvolution

      2018, Neurocomputing
      Citation Excerpt :

      New restoration methods based on learning an imaging degradation model and the characteristics of the degraded image have been extensively studied in recent years. These techniques include sparse representation-based methods [33–37] and those based on deep neural networks [38–43]. Deep neural network methods offer powerful feature learning ability, and have achieved good image restoration effects in some experiments [11].

    View all citing articles on Scopus

    Xiaoqiang Lu received the Ph.D. degree from Dalian University of Technology, Dalian, China. He is a postdoctoral researcher with the Center for Optical Imagery Analysis and Learning, State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His research interests include pattern recognition, machine learning, hyperspectral image analysis, cellular automata, and medical imaging.

    Yuan Yuan is a researcher (full professor) with Chinese Academy of Sciences, and her main research interests include visual information processing and image/video content analysis.

    Pingkun Yan received the B.Eng. degree in electronics engineering and information science from the University of Science and Technology of China, Hefei, China, and the Ph.D. degree in electrical and computer engineering from the National University of Singapore, Singapore. He is currently a full professor with the Center for Optical Imagery Analysis and Learning, State Key Laboratory of Transient Optics and Photonics, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China. His research interests include computer vision, pattern recognition, machine learning, and their applications in medical imaging.

    View full text