Nonlocal-based tensor-average-rank minimization and tensor transform-sparsity for 3D image denoising
Introduction
Compared with two-dimensional (2D) grayscale images, three-dimensional (3D) images contain richer and more redundant information, making them widely used in food safety, mineral detection, and environmental monitoring [1], [2], [3], [4]. Unfortunately, due to image transmission, sensor instability, and atmospheric influence, 3D images are inevitably polluted by Gaussian noise. The noise in 3D images not only affects the visual quality of images but also hinders subsequent applications, such as unmixing [5], [6], target detection [7], [8], and classification [9], [10], [11], [12]. Therefore, 3D image denoising is a vital topic in computer vision.
A central idea for 3D image denoising is to exploit the correlations along each dimension of 3D images. To take advantage of the correlations along the third dimension of 3D images, many low-rank matrix-based methods, which rearrange tensors into matrices, have been proposed for 3D image denoising. Under the framework of low-rank matrix recovery, Zhang et al. [13] employed the nuclear norm to describe the low-rank property of the rearranged matrices. To achieve a better low-rank approximation, Xie et al. [14] introduced a nonconvex regularizer, the weighted Schatten -norm, into the low-rank matrix approximation model. As the 3D image is a third-order tensor, the unfolding operation inevitably destroys its correlations among each dimension, and cannot fully exploit the inherent redundancy. Recently, many methods based on different tensor decompositions and the corresponding tensor rank have been proposed to capture the intrinsic low-rank property of tensors. Based on Tucker decomposition, Renard et al. [15] proposed a low-rank tensor approximation (LRTA) model that achieves both denoising and dimensionality reduction. Since Tucker decomposition considers the matricization scheme, it inevitably destroys the intrinsic structure of tensors. To reduce the adverse effect, Kilmer et al. [16] suggested the tensor singular value decomposition (t-SVD) based on the tensor-tensor product (t-product). The t-SVD has excellent performance in capturing the spatial-shifting correlations and preserving the intrinsic structures of data [17], [18], [19]. Based on the t-SVD, the tensor-multi-rank and the tensor-tubal-rank [20] are derived and have received increasing attention. As the convex surrogate of the tensor-multi-rank, the tensor nuclear norm (TNN) [20], [21], [22] was proposed and has been used in tensor recovery [23], [24], [25], [26]. Although the abovementioned methods have achieved great success in denoising, there is still much room for improving the denoising performance by exploring the redundant information of 3D images.
Based on the nonlocal self-similarity (NSS) property of 3D images, nonlocal-based methods, which stack similar blocks into groups and then denoise them separately, are proposed to further capture the redundancy of 3D images [27], [28], [29], [30], [31], [32], [33]. Considering that 3D images are composed of 2D images along the third dimension, many nonlocal-based 2D image denoising methods, such as block-matching 3D (BM3D) filtering [34] and weighted nuclear norm minimization (WNNM) [35], are applied to each band of 3D images for denoising. However, these methods neglect the correlations of 3D images along the third dimension and cannot fully exploit the NSS property of 3D images. To overcome this problem, many methods consider 3D groups as the basic denoising units. For instance, Maggioni et al. [36] proposed a filtering method that can capture both the spatial and temporal correlations of tensors. Xue and Zhao [37] introduced a novel 3D image denoising method combining the nonlocal low-rank regularization with rank-1 tensor composition to guarantee the rank uniqueness. Peng et al. [27] established a nonlocal tensor dictionary learning model (TDL) constrained by group-block-sparsity that enables similar blocks to share the same atoms from the dictionaries. Xie et al. [28] proposed a Kronecker-basis-representation (KBR) based tensor sparsity measure, which considers sparsity under Tucker and CANDECOMP/PARAFAC (CP) low-rank decompositions, and applied the measure on the groups shaped by nonlocal similar blocks. Since NSS is beneficial to exploit the redundancy and discover new correlations, the above methods gain more satisfactory denoising results.
In this paper, benefiting from the superiority of the NSS in exploiting the redundancy of 3D images, we propose a NonLocal-based denoising model combining tensor-average-Rank minimization with tensor transform-Sparsity (NLRS). (1) We use the newly proposed tensor-average-rank derived from the t-SVD to describe the low-rankness of similar groups. Compared with TNN minimization, tensor-average-rank minimization only penalizes small singular values in the optimization process. As large singular values contain more useful and important information, tensor-average-rank minimization is helpful for preserving the main components of the image (see details in Section 4.3.2). (2) As shown in Fig. 1(b1), the tensor-average-rank mainly considers the correlation along the spatial and similar dimensions of the groups, but rarely explores the correlation along the spectral dimension. To overcome this defect, we permute the third dimension of similar groups to the second dimension and use the t-linear combination to characterize the correlation along spectral dimension. By learning an orthogonal transform, the underlying clean group gains its sparse representation (see Fig. 1(b2)). Additionally, as the t-linear combination can avoid the loss of the structure caused by tensor flattening, our transform can exploit more information of underlying clean groups. In summary, the two terms characterize the correlations of underlying clean groups from different perspectives and achieve mutual promotion, which is conducive to obtaining a satisfactory denoising performance.
The main contributions of this paper can be summarized as follows:
- •
We propose a nonlocal-based denoising model that jointly employs tensor-average-rank minimization and tensor transform-sparsity to reconstruct clean 3D images. The tensor-average-rank minimization term can preserve more details and main components of similar groups, and the transform-sparsity term based on the t-linear combination can exploit more intrinsic structures of similar groups. The two terms in our model are combined organically and complement each other.
- •
We develop a proximal alternating minimization (PAM) algorithm to tackle the proposed nonconvex model and establish the global convergence guarantee. Comprehensive numerical experiments demonstrate the effectiveness of the proposed method for 3D image denoising.
Remark 1 Here, we analyze the differences among K-TSVD [38], MDTSC [39], and the proposed NLRS. The main attributes of the different methods are summarized in Table 1. First, in K-TSVD and MDTSC, the sparse model is applied to the groups stacked by overlapping patches. In the proposed model, the sparse model is applied to each similar group, which is more consistent with the sparsity hypothesis, since the groups stacked by nonlocal similar blocks have more redundancy than the groups stacked by overlapping patches. Second, K-TSVD and MDTSC are tensor synthesis dictionary learning models based on the t-linear combination. In K-TSVD, the dictionary is overcomplete, and there is no other constraint on the dictionary. In MDTSC, the author constrains the Frobenius norm of each base of the overcomplete dictionary to be one. The proposed model is a tensor transform learning model. We constrain the learned transform to be orthogonal, which means our transform is more compact and is beneficial for keeping the noise out.
The rest of this paper is organized as follows: Section 2 introduces some necessary notations and definitions. Section 3 describes the proposed model and proposes the corresponding algorithm with the convergence guarantee. Section 4 reports the experimental results and the discussions. Section 5 gives some conclusions.
Section snippets
Notations
In this paper, lowercase letters, e.g., , boldface lowercase letters, e.g., , boldface capital letters, e.g., , and boldface calligraphic letters, e.g., are used to represent scalars, vectors, matrices, and tensors, respectively. Given a third-order tensor , we denote its -th element value by . We use the Matlab notations , and to denote the -th horizontal, lateral, and frontal slice, respectively, and , and to
Proposed model and algorithm
We assume that the clean 3D data is corrupted by the additive Gaussian noise . Thus, the observed data can be expressed as
Experimental results and discussions
In this section, to evaluate the denoising performance of the proposed method on 3D images, we conduct numerical experiments on simulated and real data sets. The proposed method is compared with STROLLR [50], LRTA [15], MDTSC [39], NLTNN (using TNN under the nonlocal framework) [20], [51], NLTTNN [52], TDL [27], and KBR [28]. All parameter settings are based on the authors’ suggestions in the articles to achieve the best performance. The specific parameter selections of the proposed model are
Conculsion
In this paper, we proposed a nonlocal-based denoising model combining tensor-average-rank minimization with tensor transform-sparsity for 3D image denoising. Tensor-average-rank minimization can characterize the low-rank property of groups and preserve the main components. The transform-sparsity based on the t-linear combination can describe the correlations of the spectral dimension of similar groups, as well as maintain the intrinsic tensor structure. To optimize the proposed nonconvex model,
CRediT authorship contribution statement
Zhi-Yuan Chen: Methodology, Software, Writing – original draft. Xi-Le Zhao: Conceptualization, Validation, Resources. Jie Lin: Methodology, Validation, Writing – review & editing. Yong Chen: Formal analysis, Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research is supported by NSFC, China (No. 62131005, 61876203, 62101222,12171072), the Applied Basic Research Project of Sichuan Province, China (No. 2021YJ0107), and National Key Research and Development Program of China (No. 2020YFA0714001).
References (55)
- et al.
Variable precision rough set based unsupervised band selection technique for hyperspectral image classification
Knowl.-Based Syst.
(2020) - et al.
Endmember independence constrained hyperspectral unmixing via nonnegative tensor factorization
Knowl.-Based Syst.
(2021) - et al.
A target detection method for hyperspectral image based on mixture noise model
Neurocomputing
(2016) - et al.
Spatial-spectral classification of hyperspectral image via group tensor decomposition
Neurocomputing
(2018) - et al.
Hyperspectral image classification based on discriminative locality preserving broad learning system
Knowl.-Based Syst.
(2020) - et al.
Factorization strategies for third-order tensors
Linear Algebra Appl.
(2011) Third-order tensors as linear operators on a space of matrices
Linear Algebra Appl.
(2010)Structured low-rank approximation and its applications
Automatica
(2008)- et al.
Patched-tube unitary transform for robust tensor completion
Pattern Recognit.
(2020) - et al.
HSI-DeNet: Hyperspectral image restoration via convolutional neural network
IEEE Trans. Geosci. Remote Sens.
(2019)
Hyperspectral image denoising with total variation regularization and nonlocal low-rank tensor decomposition
IEEE Trans. Geosci. Remote Sens.
Robust thick cloud removal for multitemporal remote sensing images using coupled tensor factorization
IEEE Trans. Geosci. Remote Sens.
Hyperspectral image restoration by tensor fibered rank constrained optimization and plug-and-play regularization
IEEE Trans. Geosci. Remote Sens.
Total-variation-regularized low-rank matrix factorization for hyperspectral image restoration
IEEE Trans. Geosci. Remote Sens.
Compound rank- projections for bilinear analysis
IEEE Trans. Neural Netw. Learn. Syst.
Self-weighted robust LDA for multiclass classification with edge classes
ACM Trans. Intell. Syst. Technol. (TIST)
Hyperspectral image restoration using low-rank matrix recovery
IEEE Trans. Geosci. Remote Sens.
Hyperspectral image restoration via iteratively regularized weighted schatten -norm minimization
IEEE Trans. Geosci. Remote Sens.
Denoising and dimensionality reduction using multilinear tools for hyperspectral images
IEEE Geosci. Remote Sens. Lett.
An order- tensor factorization with applications in imaging
SIAM J. Sci. Comput.
Framelet representation of tensor nuclear norm for third-order tensor completion
IEEE Trans. Image Process.
Hyperspectral image restoration using low-rank tensor recovery
IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
Tensor robust principal component analysis with a new tensor nuclear norm
IEEE Trans. Pattern Anal. Mach. Intell.
The twist tensor nuclear norm for video completion
IEEE Trans. Neural Netw. Learn. Syst.
Mixed noise removal in hyperspectral image via low-fibered-rank regularization
IEEE Trans. Geosci. Remote Sens.
Cited by (6)
CFNet: Conditional filter learning with dynamic noise estimation for real image denoising
2024, Knowledge-Based SystemsLow-Tubal-Rank tensor recovery with multilayer subspace prior learning
2023, Pattern RecognitionRandomized sampling techniques based low-tubal-rank plus sparse tensor recovery
2023, Knowledge-Based Systems