1 Introduction

Digital images are often subjected to noise in image acquisition, transmission or storage. For this reason, the presence of noise usually decreases the accuracy of various applications in image processing; therefore, image denoising is one of the important steps in image processing and analysis. A wide range of techniques have been developed to improve the quality of images to human vision in image restoration–enhancement applications. However, denoising is an ill-conditioned problem. Noise reduction and edge preservation are the two main metrics for measuring the effectiveness of image denoising algorithms. However, it is still a challenging problem to simultaneously achieve both metrics.

Over the years, to solve the above problems, many methods have been put forward, such as Gaussian filtering, median filtering and wavelet threshold denoising. These methods are easy to implement in the existing hardware, which eliminates noise in the image and leads to missed detailed information. The denoising results cannot obtain satisfactory application demands. In recent years, a series of novel denoising algorithms with good performance have appeared, such as anisotropic filtering (AF), total variation, bilateral filtering denoising, nonlocal means denoising and 3D block matching denoising. Of these excellent algorithms, the nonlocal means (NLM) denoising algorithm [1] exhibits good denoising performance with no loss of image details, which is the concern of the majority of scholars.

The NLM denoising algorithm uses local image block (patch) redundant structure information, which is based on the image block similarity measurement criteria, by normalized structure similarity weight to obtain a noise estimate value. Because the image block can preferably describe the structure information of an image over a single pixel, the performance of the NLM denoising algorithm is better than that of other denoising methods. However, the NLM algorithm has some problems in efficiency and parameters that have seriously hindered the algorithm’s adoption in industry. Owing to the good performance of the NLM algorithm classic denoising effect, many studies aim to improve the algorithm by reducing computational complexity to achieve the purpose of the application, but reducing the number of similarity calculations between the measures of pixels is the main direction of improvement. Deledalle used a probabilistic patch-based filter to constantly update the noise estimation weight, and this approach achieved a better noise reduction effect [2]. Hu Jinrong proposed a similarity measure between pixels in the discrete cosine transform coefficients of low-frequency subspace, reducing the computational complexity of the algorithm [3]. Tasdizen proposed the main neighborhood dictionary NLM noise reduction method in which the image block is mapped to a low-dimensional subspace and a similarity measure between pixels in a low-dimensional subspace by PCA (principal component analysis), which improved denoising performance and reduced computational complexity [4]. According to the NLM parameter optimization problem algorithm, Salmon put forward the optimization of NLM key parameters (parameter smoothing kernel, similarity neighborhood size and the size of the search area), which improved the denoising effect [5]. Orchard proposed decomposition technology to reduce the dimensions so as to reduce the number of calculations by K-SVD (singular value decomposition) [2]. Hu et al. [4] proposed subspace similarity measure coefficients of low frequency between pixels in the discrete cosine transform. Zhang and Wang [5] used the Walsh–Hadamard transform of fast operation and energy concentration through similarity between metric pixels by projection onto a low-dimensional subspace to reduce calculation of the high dimensions. Mathew et al. used the image block mean and variance clustering image block to avoid calculating the weight low block of the image Euclidean distance [8]. The studies of [6, 7] project image blocks into low-dimensional space and use a similarity measure between pixels in the low-dimensional subspace through the method of PCA.

These methods, mainly through the transformation in the subspace similarity measure, reduce calculation time but also influence the effect of denoising. According to the problem of the NLM algorithm, this paper proposes reducing the speed while preserving improved texture detail information. Aiming at the problem of the high computational complexity of the algorithm, a novel and fast nonlocal means (NLM) image denoising method with fuzzy theory and a structure tensor is proposed to address the problem of the high computing cost of similarity weights. Compared to the effect with other improved NLM algorithms, the denoising edge effect is improved by PNSR and SSIM, and the results are reduced computational complexity and enhanced denoising performance.

The remainder of this paper is organized as follows: Section 2 introduces the nonlocal means algorithm and describes how to denoise an image. Section 3 explains the concept of fuzzy metrics and structure tensor and builds visual similarity by fuzzy metrics and a structure tensor, which constitutes an improved denoising algorithm of visual features based on fuzzy metrics. The experimental results of comparing the method in this paper with conventional methods are presented in Sect. 4. Finally, concluding remarks are included in Sect. 5.

2 State of the art

The general additive noise image denoising model can be defined by

$$ v(n,m) = u(n,m) + \gamma (n,m) $$
(1)

where (n, m) represents the value of pixels in 2D image space coordinates, v(n, m) represents the pixels of the image with noise pollution, for which the coordinate is (n, m), u(n, m) represents the real image, and γ(n, m) represents the noise image with Gaussian additive white noise.

For two-dimensional images with a size of N × M, in the NLM denoising algorithm, the estimation value of u(n, m) is defined as:

$$ u[f][n,m] = \sum\limits_{k = 0}^{N - 1} {\sum\limits_{l = 0}^{M - 1} {w[n,m;k,l]} } \times f[k,l] $$
(2)

where \( w[n,m;k,l] = \frac{1}{Z[n,m]}e^{{ - \frac{{\left\| {N[{\rm P}_{n,m} ] - N[{\rm P}_{k,l} ]} \right\|_{2,a}^{2} }}{h}}} \) and \( Z[n,m] = \sum_{k = 0}^{N - 1} {\sum_{l = 0}^{M - 1} {e^{{ - \frac{{\left\| {N[{\rm P}_{n,m} ] - N[{\rm P}_{k,l} ]} \right\|_{2,a}^{2} }}{h}}} } } \)

\( w[n,m;k,l] \) is the similarity coefficient of weight u(k, l) and u(n, m), z[n, m] is the normalization factor and w[n, m; k, l]ϵ[0, 1], Pn,m represents the image block, the center is (n, m), the size is (2b + 1) × (2b + 1), h is the control degree of the attenuation exponential function, which is generally set to σ multiple noise of the standard variance, and the gray value of each image block is the similarity between vectors measured by a Gaussian-weighted Euclidean distance.

For estimation u[f][n, m] of each image pixel point to obtain the denoising results. From the formula of the NLM algorithm, the more similar the image blocks (n, m) and image block (k, l) are, the higher the weights used for denoising estimation are. The values of image pixels require all points of the image in an image block to calculate the weight. Due to the structural similarity of image blocks with spatial concentration, in order to reduce the computational complexity, this algorithm limited the search scope to s region that is the center of (n, m). The time complexity is reduced to O(NMS2B2). In the process of denoising, each pixel of image blocks obtains weights by a Gaussian-weighted Euclidean distance to compute the similarity with all other image blocks in the search window, leading to high computational complexity of the algorithm. It is thus difficult to apply to the actual project.

3 Methodology

It is important to realize the good performance of the NLM algorithm by a reasonable choice of the similarity measure. Aiming at the problems existing in NLM, this paper first employs the structure tensor to extract texture descriptions that divide the image into smooth regions, edge regions and texture regions and then uses a similarity measure with visual features based on gray theory to obtain the denoising results, which reduces the running time.

3.1 Local structure tensor

The local structure information of the edge and gradient can be used to describe the image reasonably, but directly calculating the image gradient or edge is not accurate in the noise interference; the local structure tensor can more finely characterize the structure characteristic of the image edge.

For a given image f, the gradient is g = ∇f(a, b), the image of a pixel f (a, b) of the local gradient vector for G = (G1, G2,…, Gn) in the field of K × K block (n), and the local structure tensor is defined as follows:

$$ J = GG^{\text{T}} = \left[ {\begin{array}{*{20}c} {\sum\limits_{i \in n} {g_{{a_{i} }}^{2} } } & {\sum\limits_{i \in n} {g_{{a_{i} }}^{{}} g_{{b_{i} }}^{{}} } } \\ {\sum\limits_{i \in n} {g_{{a_{i} }}^{{}} g_{{b_{i} }}^{{}} } } & {\sum\limits_{i \in n} {g_{{b_{i} }}^{2} } } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {J_{11} } & {J_{12} } \\ {J_{21} } & {J_{22} } \\ \end{array} } \right] $$
(3)

SVD decomposition of the covariance matrix can be obtained by:

$$ J = W\left[ {\begin{array}{*{20}c} {\lambda_{1}^{'} } & 0 \\ 0 & {\lambda_{2}^{'} } \\ \end{array} } \right][\begin{array}{*{20}c} {R_{1} } & {R_{2} } \\ \end{array} ] $$
(4)

where W⊥R, the feature vector R1 is the main direction of the image structure, R2 is the secondary direction of the image structure and R1⊥R2 characteristic values, \( \lambda_{1}^{'} \) and \( \lambda_{2}^{'} \) represent the local structure tensor of the energy change in the R1 and R2 directions, and the image block rotates at an arbitrary angle. The characteristics of the local structure tensor image value \( \lambda_{1}^{'} \) and \( \lambda_{2}^{'} \) will not change.

To define the texture description scalar to also reflect the image block it characterizes, the selection of the neighborhood block size and the NLM image block should be consistent.

The definition of a new edge and texture feature description function F(x) can be obtained by:

$$ F(x) = \frac{{(\lambda_{1}^{'} - \lambda_{2}^{'} )^{n} }}{{1 + B(\lambda_{1}^{'} + \lambda_{2}^{'} )}} \times \lambda_{1}^{'} $$
(5)

where B is the chosen constant, n = 1, 2. F(x) describes the geometric structure in image intensity information. In the image information flat area, indicating no image structure information, its value is small.

In the image edge area, its value is higher, when the first item value of the edge strength with great energy is greater by \( \lambda_{1}^{'} \approx \lambda_{2}^{'} > 0 \). In the texture area, its value tends to the middle.

3.2 Image region partition

This section uses two k-means clustering and edge texture feature description functions, which divide the entire image into smooth areas, edge areas and texture areas. In the similarity computation, this function excludes nonhomogeneous image blocks, avoids unnecessary calculations and improves the operation speed of the algorithm.

The two k-means algorithms are the improved k-means clustering algorithm, which is unable to determine the initial centroid [9]. The main process uses the k-means algorithm to divide all the data into two clusters and then continues to use the k-means algorithm for splitting operations from one cluster until a cluster of partitions has been established. The clustering of values of F(x) uses two k-means to separate texture edges and flat areas first by the threshold T1 and again to separate the texture and edge pixels by the threshold T2. The original algorithm search window is amended as:

$$ s_{i} = \left\{ {\begin{array}{*{20}l} {F(s_{i} ) < T_{1} } \hfill & \quad {{\text{smooth}}\;{\text{area}}} \hfill \\ {F(s_{i} ) > T_{2} } \hfill &\quad {{\text{edge}}\;{\text{area}}} \hfill \\ {F(s_{i} ) > T_{1} \cap F(s_{i} ) < T_{2} } \hfill & \quad {{\text{texture}}\;{\text{area}}} \hfill \\ \end{array} } \right. $$
(6)

According to the edge texture feature description of value F(si), f(x) belongs to the cluster of position i in order to obtain the corresponding threshold value, to filter out the other two areas and to accelerate calculation.

3.3 Visual feature similarity

Fuzzy topology has become an area of active research in the previous decades and is a fundamental and important branch of fuzzy theory due to its wide range of applications. Obtaining an appropriate concept of fuzzy metrics in fuzzy topology is one of the most paramount problems. Fuzzy metrics are useful for various applications according to many previous studies. Fuzzy metrics instead of classical metrics in particular have been applied to color image filtering to improve some filters.

An example of fuzzy metric M on a set X with the help of continuous t-norms was introduced and studied by George and Veeramani [10].

The following are two main advantages of the fuzzy metric:

  1. 1.

    The outcome given by M is in the space [0, 1] so that the nature of the distance metric being computed is of no concern.

  2. 2.

    The M values are perfectly consistent in other fuzzy methods, as the result obtained by M can be directly applied as a fuzzy certainty degree.

Assuming a continuous t-norm t as a binary operation, [0, 1] × [0, 1] → [0, 1], X is a nonempty set satisfying the following five requirements (R1 to R5) for all x, y∈X, \( X \in [a,b] \), \( G > \left| a \right| > 0 \), \( \alpha > 0 \) and l = 1, 2…, we define \( H_{l}^{\alpha } \) by:

$$ H_{l}^{\alpha } ({\mathbf{x}},{\mathbf{y}},t) = \prod\limits_{i = 1}^{l} {\left( {\frac{{\hbox{min} \{ x_{i} ,y_{i} \} + G}}{{\hbox{max} \{ x_{i} ,y_{i} \} + G}}} \right)}^{\alpha } $$
(7)

where x = (x1,…,xl), y = (y1,…,yl), and t > 0. \( H_{l}^{\alpha } ({\mathbf{x}},{\mathbf{y}},t) \) is considered the degree of nearness from x to y with respect to t, which is also called the fuzzy metric. The pulse noise reduction filter of Ref. [11] integrating the fuzzy metric and fast similarity has high computing performance and a good filtering effect. We combine two different fuzzy metrics into a unique fuzzy metric.

Although NLM has achieved a good denoising effect, it is not enough to consider the structural information of image patches, which leads to the lack of structural information for the original image. To highlight the importance of edge information in the image structure, we combine the fuzzy metric propositions described above to obtain the simplified fuzzy metric function that emphasizes the edge of the image patches.

when l = 1 and α = 1, setting \( y = {\bar{\mathbf{x}}}_{w} = \frac{1}{{p^{2} }}\sum\nolimits_{j = 1}^{{p^{2} }} {x_{j} } \).

Formula (7) is simplified as

$$ H_{{x_{i} }} = H(x_{i} ,\overline{{{\mathbf{x}}_{w} }} ,t) = \left( {\frac{{\hbox{min} \{ x_{i} ,\overline{{{\mathbf{x}}_{w} }} \} + t}}{{\hbox{max} \{ x_{i} ,\overline{{{\mathbf{x}}_{w} }} \} + t}}} \right) $$
(8)

The fuzzy metric of the image is obtained by Formula (8), and the range of \( H_{{x_{i} }} \) is [0, 1], corresponding to the gray value image t = 255.

4 An improved NLM using structure tensor

A similarity weighting method using a fuzzy metric is proposed in this section; this method discards small similarity weights of the image patches with a structure tensor and enhances the similarity coefficients. This method is more beneficial with respect to utilizing image structure information.

To ensure the high similarity of image patches to obtain high weights, the improved estimation of the NLM image denoising algorithm is described by:

$$ SF[f[m,n]] = \sum\limits_{{}} {\psi (f[m,n],f[k,l])f[k,l]} $$
(9)

where \( \psi (f[m,n],f[k,l]) \) is the weight of image patch f[m, n] and f[k, l] is in the neighbor pixels S. It satisfies \( 0 \le \psi (f[m,n],f[k,l]) \le 1 \).

$$ \psi [n,m;k,l] = K_{\text{Flat}} \times e^{{ - \frac{{\left\| {H(x_{m,n} ,\overline{{{\mathbf{x}}_{w} }} ,t) - H(x_{k,l} ,\overline{{{\mathbf{x}}_{w} }} ,t)} \right\|_{2,a}^{2} }}{h}}} $$
(10)

where KFlat is the smoothing kernel function and \( \bar{X}_{w} = \frac{1}{s}\sum_{j = 1}^{s} {x_{j} } \).

To prevent the lower similarity of image patches from participating in weight allocation, the mean value of the fuzzy similarity of the neighborhood is used as the threshold.

$$ K_{\text{Flat}} = \left\{ {\begin{array}{*{20}l} {F(s)} \hfill & {F(s) \ge \overline{F(s)} } \hfill \\ 0 \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right. $$
(11)

where \( \overline{F(s)} \) is the mean value of the edge and texture, the feature description function with a local structure tensor, and S is the neighbor pixel window.

The proposed algorithm mainly contains two parts: runtime fuzzy metric of the pixel and visual feature similarity weight. The complexity of the fuzzy metric of the pixel is of a linear complexity O(N). Therefore, the efficiency of the algorithm was obviously improved.

5 Experimental results and analysis

5.1 Test condition

To demonstrate the effectiveness of the improved approach, we selected two 256 × 256 pixel images as test images, as shown in Fig. 1: a remote sensing image G1 and the Barbara, Pepper and Lena images.

Fig. 1
figure 1

Test images. a Remote sensing image G1. b Lena image

Full size image

The improved approach is compared with FM-PatchGP [12], which represents the feature-based NLM denoising method; BM3D [14], which represents an excellent denoising method; and K-SVD-NLM [15] and DCT-NLM [16], which constitute an excellent and improved NLM denoising method based on performance and computational time. The qualities of several methods are compared in terms of the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR).

The structural similarity index is a new index for evaluating the quality of images that uses brightness, contrast and structural similarity as three comprehensive indexes as a final objective image quality evaluation value for the new index [16]. Its range of values is [0 ~ 1]. Meanwhile, the closer the its value is to 1, the more similar the two images are, and when the images are identical, the value of SSIM is 1.

$$ {\text{SSIM(}}x , { }y )= \frac{{ ( 2\mu_{x} \mu_{y} + A_{ 1} ) ( 2\sigma_{xy} + A_{ 2} )}}{{(\mu_{x}^{2} + \mu_{y}^{2} + A_{ 1} )(\sigma_{x}^{2} + \sigma_{y}^{2} + A_{ 2} )}} $$
(12)

where \( \mu_{x} \), \( \sigma_{x} \) and \( \sigma_{xy} \) represent the mean, the variance of x and the covariance of x with y, respectively. \( \mu_{x} \) and \( \sigma_{x}^{{}} \) are the evaluation indexes of brightness and contrast, respectively. \( \sigma_{xy} \) is evaluated for the corresponding structural changes in x and y. A1 and A2 are smaller positive constants. The values range from 0 to 1 for estimating the two images’ similarity index, where the best value, 1, is achieved if and only if \( \sigma_{x}^{{}} = \sigma_{y}^{{}} \).

5.2 Tuning of the parameters

To show the effectiveness of the improved approach, the appropriate settings for the improved approach are important. After the simulation, we found that the parameter set (α, β) = (1, 1) is the best selection. We then needed to determine the optimal range of search area R and image patch size S.

Reference [7] analyzed the search area set to 21 × 21, which can achieve good performance in the two aspects of time efficiency and noise reduction. Therefore, the search range R is set to 21 in this paper’s experiments. The larger the image patch is, the more abundant the structure is, but the patch may be too large to have similarities among patches, leading to lower weights. To obtain the optimal image patch size, the noise images with standard deviations σ = {10, 15, 20, 25, 30, 35, 40} are evaluated by PSNR. When S = 3, 5, 7, 9, with an increase in structure information, the performance of the denoising method obviously increases. However, when S = 11, 13, 15, as the image patch size increases, the value of PSNR decreases slightly. The optimal value of S is 9.

5.3 Performance analysis

To illustrate the performance of the proposed method, some experimental results are provided in this section. The proposed method and FM-PatchGP are implemented using MATLAB 2010b, and the source code of K-SVD-NLM, DCT-NLM and BM3D is from the author’s website. The results of PSNR and SSIM with different denoising techniques for remote sensing noise images G1 and Lena are shown in Figs. 2 and 3, where the same parameters were used.

Fig. 2
figure 2

Comparison of the improved algorithm with K-SVD-NLM, DCT-NLM, FM-PatchGP, cNLM and BM-3D by the PSNR index (the noise images with standard deviations σ = {10, 15, 20, 25, 30, 35, 40}). a The index value of G1 in PNSR. b The index value of Lena in PNSR

Full size image
Fig. 3
figure 3

Comparison of the improved algorithm with K-SVD-NLM, DCT-NLM, FM-PatchGP, cNLM and BM-3D by the SSIM index (the noise images with standard deviations σ = {10, 15, 20, 25, 30, 35, 40}). a The index value of the G1 image in SSIM. b The index value of the Lena image in SSIM

Full size image

Figures 2 and 3 show the comparison of six algorithms on the noisy images with standard deviations σ = {10, 15, 20, 25, 30, 35, 40} by values of PSNR and SSIM. As shown in Fig. 2, the overall index of DCT-NLM is better, and as it increases, the value for the improved algorithm is close to that for DCT-NLM. As shown in Fig. 4, the value of SSIM decreases rapidly with an increasing noise level. Conventional methods have been subjected to some edge detail smoothing.

Fig. 4
figure 4

The PSNR values and results for the Lena image (a is the result of cNLM, b is the result of K-SVD-NLM, c is the result of DCT-NLM, d is the result of BM3D, and e is the result of the improved algorithm proposed in this paper). a PSNR = 30.14, b PSNR = 31.12, c PSNR = 32.21, d PSNR = 32.10, e PSNR = 32.46

Full size image

The proposed method is supported by clear images with detail preservation and edge sharpness. It is clear from these visual results that, compared with conventional methods, the proposed NLM method based on fuzzy metrics has the best quality since it can reduce artifacts by fuzzy metrics to measure the subtle changes in the image structure.

The details shown in Fig. 4 show that the PSNR value of our proposed improved algorithm is better than that achieved using the K-SVD-NLM, DCT-NLM and cNLM algorithms. There are more edges in the detailed region of our proposed algorithm than in the other algorithms.

Table 1 shows the results of comparing PSNR and SSIM in four different algorithms. The improved algorithm is better than the cNLM algorithm in terms of PSNR and SSIM. The noise performance for the improved algorithm and DCT-NLM and K-SVD-NLM is higher than that for cNLM. The texture detail-rich Barbara and Lena image PSNR and SSIM values are more obvious than those for cNLM and K-SVD-NLM.

Table 1 Performance comparison of the proposed method, cNLM, DCT-NLM and K-SVD-NLM on the PSNR and SSIM
Full size table

The residual noise (RN) component is the difference between the denoised image and the noisy image. As the different algorithms have different denoising effects, it also takes part of the image information into the RN component, and the amount of RN information brought by the image information can also be used as a subjective quality evaluation. Figures 4 and 5 show a detailed comparison of RN images of the Pepper image and the Lena image, respectively, with different methods. Note that the less structural information on the RN image is preserved, the better the performance. The results from LG-NLM and BM3D tended to contain more detailed image structural information than the other methods did. This finding illustrates that LG-NLM and BM3D lost considerable image structure information from the Pepper image. RN images from LG-NLM, LR-NLM, NLM and BM3D still preserve part of the clear structural information in the head region of the Lena image.

Fig. 5
figure 5

Comparison of residual noise images in denoising the Lena image with different denoising methods. a Noising Lena, b proposed method, c NLM, d LG-NLM, e BM3D, f LR-NLM

Full size image

To validate the proposed algorithm, we conducted simulations using MATLAB with an Intel(R) Core(TM) i5 CPU M460 @ 2.82GHZ with 3 GB memory.

Finally, we compare execution times. Comparisons of the computation times of our proposed NLM, K-SVD-NLM, FM-PatchGP, DCT-NLM and BM3D are shown in Table 2 using four images, each of different sizes. The table shows that the processing times for our proposed NLM are very low. This is due to the measure of the structure tensor that eliminates a large number of pixels of low similarity by the similarity index of the visual feature by fuzzy metrics. The execution time for the structure tensor of our proposed NLM is lower than that for the other NLM for the Euclidean distance calculation. FM-PatchGP is fast; however, as the image size increases, the execution time for the FM-PatchGP is significantly longer than that for the proposed NLM.

Table 2 Comparison of denoising execution time (unit: second)
Full size table

6 Conclusions

To solve the problem of the high computational cost of the similarity weight, a novel and fast denoising algorithm based on structure tensor and fuzzy metrics is proposed. A structure tensor is adaptively applied to measure the similarity of the image patches and filter out some less similar image patches. The improved NLM version can not only maintain the image edge and texture but also improve the adaptability of the algorithm in dealing with different images. Compared to the existing state-of-the-art approaches, the proposed method exhibited improved performance in terms of PSNR and SSIM indexes. The method also demonstrated better generalization performance than its nearest competitors, and its performance was competitive.