Nonlocal-based tensor-average-rank minimization and tensor transform-sparsity for 3D image denoising

Abstract

Three-dimensional (3D) image denoising is an essential problem of low-level computer vision tasks, and it is also a key preprocessing step for subsequent applications. Recently, nonlocal-based models have received increasing attention in the field of 3D image denoising because they are powerful in promoting the redundancy of 3D images. In this paper, we propose a nonlocal-based denoising model combining tensor-average-rank minimization with tensor transform-sparsity (NLRS) to exploit the redundancy of each similar group. In the proposed model, we employ tensor-average-rank minimization to exploit the low-rankness of the underlying clean similar groups. Moreover, we introduce a tensor-based transform learning so that each underlying clean similar group gains sparse representation under the corresponding transform. The low-rank term and the transform-sparsity term are complementary in terms of noise suppression and information preservation. To tackle the nonconvex model, we develop a proximal alternating minimization (PAM) algorithm and theoretically prove the convergence of the algorithm. Numerical experiments on simulated and real data sets are conducted to demonstrate the effectiveness of the proposed method for 3D image denoising.

Introduction

Compared with two-dimensional (2D) grayscale images, three-dimensional (3D) images contain richer and more redundant information, making them widely used in food safety, mineral detection, and environmental monitoring [1], [2], [3], [4]. Unfortunately, due to image transmission, sensor instability, and atmospheric influence, 3D images are inevitably polluted by Gaussian noise. The noise in 3D images not only affects the visual quality of images but also hinders subsequent applications, such as unmixing [5], [6], target detection [7], [8], and classification [9], [10], [11], [12]. Therefore, 3D image denoising is a vital topic in computer vision.

A central idea for 3D image denoising is to exploit the correlations along each dimension of 3D images. To take advantage of the correlations along the third dimension of 3D images, many low-rank matrix-based methods, which rearrange tensors into matrices, have been proposed for 3D image denoising. Under the framework of low-rank matrix recovery, Zhang et al. [13] employed the nuclear norm to describe the low-rank property of the rearranged matrices. To achieve a better low-rank approximation, Xie et al. [14] introduced a nonconvex regularizer, the weighted Schatten $p$-norm, into the low-rank matrix approximation model. As the 3D image is a third-order tensor, the unfolding operation inevitably destroys its correlations among each dimension, and cannot fully exploit the inherent redundancy. Recently, many methods based on different tensor decompositions and the corresponding tensor rank have been proposed to capture the intrinsic low-rank property of tensors. Based on Tucker decomposition, Renard et al. [15] proposed a low-rank tensor approximation (LRTA) model that achieves both denoising and dimensionality reduction. Since Tucker decomposition considers the matricization scheme, it inevitably destroys the intrinsic structure of tensors. To reduce the adverse effect, Kilmer et al. [16] suggested the tensor singular value decomposition (t-SVD) based on the tensor-tensor product (t-product). The t-SVD has excellent performance in capturing the spatial-shifting correlations and preserving the intrinsic structures of data [17], [18], [19]. Based on the t-SVD, the tensor-multi-rank and the tensor-tubal-rank [20] are derived and have received increasing attention. As the convex surrogate of the tensor-multi-rank, the tensor nuclear norm (TNN) [20], [21], [22] was proposed and has been used in tensor recovery [23], [24], [25], [26]. Although the abovementioned methods have achieved great success in denoising, there is still much room for improving the denoising performance by exploring the redundant information of 3D images.

Based on the nonlocal self-similarity (NSS) property of 3D images, nonlocal-based methods, which stack similar blocks into groups and then denoise them separately, are proposed to further capture the redundancy of 3D images [27], [28], [29], [30], [31], [32], [33]. Considering that 3D images are composed of 2D images along the third dimension, many nonlocal-based 2D image denoising methods, such as block-matching 3D (BM3D) filtering [34] and weighted nuclear norm minimization (WNNM) [35], are applied to each band of 3D images for denoising. However, these methods neglect the correlations of 3D images along the third dimension and cannot fully exploit the NSS property of 3D images. To overcome this problem, many methods consider 3D groups as the basic denoising units. For instance, Maggioni et al. [36] proposed a filtering method that can capture both the spatial and temporal correlations of tensors. Xue and Zhao [37] introduced a novel 3D image denoising method combining the nonlocal low-rank regularization with rank-1 tensor composition to guarantee the rank uniqueness. Peng et al. [27] established a nonlocal tensor dictionary learning model (TDL) constrained by group-block-sparsity that enables similar blocks to share the same atoms from the dictionaries. Xie et al. [28] proposed a Kronecker-basis-representation (KBR) based tensor sparsity measure, which considers sparsity under Tucker and CANDECOMP/PARAFAC (CP) low-rank decompositions, and applied the measure on the groups shaped by nonlocal similar blocks. Since NSS is beneficial to exploit the redundancy and discover new correlations, the above methods gain more satisfactory denoising results.

In this paper, benefiting from the superiority of the NSS in exploiting the redundancy of 3D images, we propose a NonLocal-based denoising model combining tensor-average-Rank minimization with tensor transform-Sparsity (NLRS). (1) We use the newly proposed tensor-average-rank derived from the t-SVD to describe the low-rankness of similar groups. Compared with TNN minimization, tensor-average-rank minimization only penalizes small singular values in the optimization process. As large singular values contain more useful and important information, tensor-average-rank minimization is helpful for preserving the main components of the image (see details in Section 4.3.2). (2) As shown in Fig. 1(b1), the tensor-average-rank mainly considers the correlation along the spatial and similar dimensions of the groups, but rarely explores the correlation along the spectral dimension. To overcome this defect, we permute the third dimension of similar groups to the second dimension and use the t-linear combination to characterize the correlation along spectral dimension. By learning an orthogonal transform, the underlying clean group gains its sparse representation (see Fig. 1(b2)). Additionally, as the t-linear combination can avoid the loss of the structure caused by tensor flattening, our transform can exploit more information of underlying clean groups. In summary, the two terms characterize the correlations of underlying clean groups from different perspectives and achieve mutual promotion, which is conducive to obtaining a satisfactory denoising performance.

The main contributions of this paper can be summarized as follows:

• We propose a nonlocal-based denoising model that jointly employs tensor-average-rank minimization and tensor transform-sparsity to reconstruct clean 3D images. The tensor-average-rank minimization term can preserve more details and main components of similar groups, and the transform-sparsity term based on the t-linear combination can exploit more intrinsic structures of similar groups. The two terms in our model are combined organically and complement each other.

• We develop a proximal alternating minimization (PAM) algorithm to tackle the proposed nonconvex model and establish the global convergence guarantee. Comprehensive numerical experiments demonstrate the effectiveness of the proposed method for 3D image denoising.

Remark 1

Here, we analyze the differences among K-TSVD [38], MDTSC [39], and the proposed NLRS. The main attributes of the different methods are summarized in Table 1. First, in K-TSVD and MDTSC, the sparse model is applied to the groups stacked by overlapping patches. In the proposed model, the sparse model is applied to each similar group, which is more consistent with the sparsity hypothesis, since the groups stacked by nonlocal similar blocks have more redundancy than the groups stacked by overlapping patches. Second, K-TSVD and MDTSC are tensor synthesis dictionary learning models based on the t-linear combination. In K-TSVD, the dictionary is overcomplete, and there is no other constraint on the dictionary. In MDTSC, the author constrains the Frobenius norm of each base of the overcomplete dictionary to be one. The proposed model is a tensor transform learning model. We constrain the learned transform to be orthogonal, which means our transform is more compact and is beneficial for keeping the noise out.

The rest of this paper is organized as follows: Section 2 introduces some necessary notations and definitions. Section 3 describes the proposed model and proposes the corresponding algorithm with the convergence guarantee. Section 4 reports the experimental results and the discussions. Section 5 gives some conclusions.