A multi-parameter regularization model for image restoration

Highlights

• Based on total variation and wavelet frame, a multi-parameter regularization model for image restoration is proposed.

• An effective algorithm based on ADMM is given for solving the new model, i.e., TVframe.

• Convergence analysis of the new algorithm is given.

• Numerical experiments show that TVframe outperforms several state-of-the-art image restoration approaches.

Abstract

This paper presents a new multi-parameter regularization model for image restoration (IR) based on total variation (TV) and wavelet frame (WF). On one hand, the Rudin–Osher–Fatemi (ROF) model using TV as the regularization term has been proven to be very effective in preserving sharp edges and object boundaries which are usually the most important features to recover. On the other hand, adaptively exploiting the regularity of natural images has led to the successful WF approaches for IR. In this paper, we propose a novel model that combines these two approaches together to restore images from blurry, noisy and partial observations. Computationally, we use the alternative direction method of multiplier (ADMM) to solve the new model and provide its convergence analysis in the appendix. Numerical experiments on a set of IR benchmark problems show that the proposed model and algorithm outperform several state-of-the-art approaches in terms of the restoration quality.

Introduction

Restoration from noisy, blurry and partial observations is a fundamental task in image processing (IP) ranging from computer sciences, electronic engineering and remote sensing to biology and medical sciences. Mathematically, the observation model is as follows:

$$y=\mathit{Kx}+\eta$$

where y, $x\in {\mathbb{R}}^{M\times N}$ are observed and target images, respectively, K is a linear corruption operator that maps ${\mathbb{R}}^{M\times N}$ into ${\mathbb{R}}^{M\times N}$, and $\eta \in {\mathbb{R}}^{M\times N}$ is the zero-mean white Gaussian noise with variance σ². Image restoration (IR) is an inverse problem aiming at recovering x from the corrupted observation image y. There are several approaches to tackle this issue, to name just a few, statistics [1], Fourier or wavelet transforms [2], and variational analysis [3], [4].

It is well known that the process to recover x from y is ill-posed in the sense that the restoration results may not be continuously dependent on the observation error [4]. A widely used method to handle this problem is resorting to regularization, where the underlying image x is approximated by the solution of

$$\underset{x}{\min }\frac{1}{2}{\parallel y-\mathit{Kx}\parallel }_2^2+\lambda R\left(x\right).$$

The first term in (2) incorporates the measurement y and the second term is the regularization term which contains the prior information of the underlying solution. The regularization parameter $\lambda >0$ controls the tradeoff between these two terms.

Classical Tikhonov regularization [5] using $R\left(x\right)=\parallel \mathit{Dx}{\parallel }_2^2$ with D to be a local difference operator, however, tends to make images over smooth and fail to preserve sharp edges. The ROF model [6], [7] using total variation (TV) as a regularizer has been shown a great success in preserving the most important features in images such as sharp edges and object boundaries. However, the well known drawback of TV based methods is to create staircasing phenomenon in the flatten zones and small object corners and lead to the loss of image contrasts [8], [9], [10], [11].

There are several higher-order variational regularization based models that can suppress the staircasing effect created by TV-based models. The first type is replacing the TV term with higher-order regularizes directly [12], [13], [14], [15] to keep the smooth parts of the image. The second category uses multi-parameter regularization models based on TV and other regularizer that incorporates the high order TV [16] or curvature information of the level set of the target image [17], [18]. Another kind is treating the underlying image as the sum of cartoon part measured by the TV norm while the flatten part measured by a higher-order norm, such as the inf-convolution TV based model [3], the fourth order based models [10], [11], the total generalized variation model [19], and the two framelet-based models [20]. The models mentioned above can remedy the drawbacks of TV based model to some extent. However, it also needs some efforts to improve them both theoretically and computationally. Mathematically, the first two kinds of approaches assume higher regularities prior of the target image which may exclude the edges unfortunately. Numerically, it is quite challenging to develop fast solvers with theoretical guarantee to minimize the energy functionals involving high order derivatives since the related Euler–Lagrange equations are often fourth order.

Sparse representation is another hot topic in IR and other IP fields [21]. In general, most natural images are usually (approximately) sparse under some dictionaries, e.g., Fourier, wavelet, contourlet, curvelet, and WF. In order to make full use of the sparsity prior information, the redundant dictionaries such as WF are preferred in the process of IP. Due to that different framelets have different orders of vanishing moment, the magnitude of framelet coefficients of the image can adaptively indicate the regularity of the underlying image. Therefore, the framelet-based synthesis, analysis, and balanced variational models can significantly improve IR quality [20], [22], [23], [24], [25], [26], [27], [28], [29], [30]. The authors in [30] proposed a new scheme, which simultaneously maximizes the sparsity of the blur kernel and the sparsity of the clear image, i.e., using the curvelet system for the blur kernel while framelet system for the latter. Recently, it has been shown that several important variational PDE-based image regularization models can be interpreted as a certain kind of framelet regularization with specified WFs [20], [31], [32]. Also, there are many other items that can be used to improve the quality of the IR algorithms, such as the nonlocal information of the image [33], coupled dictionary [34], and multiple images [35].

Inspired by the idea that framelet transforms play a role of multi-scale differential operator adaptively, we propose a multi-parameter regularization IR model based on TV and framelet to preserve sharp edges and to avoid staircasing simultaneously. To be precise, we use TV regularizer to keep sharp edges and choose a proper framelet whose coefficient can adaptively detect higher regularity zones and then can measure the smoothness of the underlying image, which is the first main contribution of this paper. We should point out that our newly proposed model is not only different from the ones aiming at decomposing the original image into two parts and using different regularization term to measure them [3], [10], [11], [19], [20] but also different from the models utilizing multi-parameter regularization combined TV with higher-order regularizer [16], [17], [18]. The authors in [20] give a scheme for restoring image, which simultaneously use total variation and framelets to be the regularization schemes. The difference between our algorithm and this scheme is that during their method, the underlying image is firstly divided into two parts, then enforce two regularization items to the remaining two parts while our algorithm does not need to divide the image, instead of which, total variation and framelets are directly acting on the whole image in TV frame. We also noticed that there are several works that hybrid TV or higher-order TV with certain wavelet regularization to preserve image details and textures [27], [28], [36], [37], [38], [39]. Our newly proposed TV-analysis based model also differs from them since they are developed from different motivations using different wavelets or just use the synthesis model.

Numerically, we adopt the alternative direction method of multiplier (ADMM) to develop a fast and stable algorithm and establish its convergence analysis, which is the second main contribution of this paper. The ADMM is first introduced in [40], [41] and is been shown to have the advantages of notable stability and high rate of convergence, as a result of which, it is now widely used for the minimization of convex functionals under linear-equality constraints. Here, similar to [42], we use the ADMM as a single and efficient tool to derive the explicit reconstruction algorithm for IR. After bringing in two auxiliary variables, we first transform this model into a new constrained problem and then use the variable splitting technique to transform the resulting constrained problem into a different unconstrained problem. The obtained unconstrained problem is then solved by the ADMM efficiently. It is shown in the simulation experiments that the newly proposed algorithm is very effective and efficient in IR.

The rest of this paper is organized as follows. In Section 2, we review the WF system and image representation. The proposed new model based on framelet and TV is introduced in Section 3, along with the corresponding ADMM algorithm in detail. In Section 4, we conduct several numerical experiments to demonstrate that both the new model and the ADMM algorithm are effective and efficient. Concluding remarks and future works are discussed in Section 5 while the proof of the convergence theorem of our newly proposed algorithm is given in Appendix A.