The Optimal Weights of Non-local Means for Variance Stabilized Noise Removal

Abstract

The Non-Local Means (NLM) algorithm is a fundamental denoising technique widely utilized in various domains of image processing. However, further research is essential to gain a comprehensive understanding of its capabilities and limitations. This includes determining the types of noise it can effectively remove, choosing an appropriate kernel, and assessing its convergence behavior. In this study, we optimize the NLM algorithm for all variations of independent and identically distributed (i.i.d.) variance-stabilized noise and conduct a thorough examination of its convergence behavior. We introduce the concept of the optimal oracle NLM, which minimizes the upper bound of pointwise $L_{1}$ or $L_{2}$ risk. We demonstrate that the optimal oracle weights comprise triangular kernels with point-adaptive bandwidth, contrasting with the commonly used Gaussian kernel, which has a fixed bandwidth. The computable optimal weighted NLM is derived from this oracle filter by replacing the similarity function with an estimator based on the similarity patch. We present theorems demonstrating that both the oracle filter and the computable filter achieve optimal convergence rates under minimal regularity conditions. Finally, we conduct numerical experiments to validate the performance, accuracy, and convergence of $L_{1}$ and $L_{2}$ risk minimization for NLM. These convergence theorems provide a theoretical foundation for further advancing the study of the NLM algorithm and its practical applications.

Appendix. Proofs of the Main Results

1.1 The Proof of Theorem 1

Because the objective function ${\mathcal {J}}_{k,\phi }(\omega )$ defined by (12) is continuously differentiable, and both the equality constraint $\sum _{x\in {\mathcal {N}}_{x_{0},D}}\omega (x)=1$ and inequality constraints $\omega (x)\ge 0, x \in {\mathcal {N}}_{x_{0},D}$ are differentiable, we consider the Lagrange function:

$$\begin{aligned} {\mathcal {L}} (\omega ,a(x_{0}),g) = {\mathcal {J}}_{k,\phi }(\omega ) -k a(x_{0})\left( \sum _{x\in {\mathcal {N}}_{x_{0},D}}\omega (x)-1 \right) -k\sum _{x\in {\mathcal {N}}_{x_{0},D}}g(x)\omega (x), \end{aligned}$$

where $k=1,2$, $\omega , g \in {\mathbb {R}}^{M}$ are vectors whose components are $\omega (x)$, $x \in {\mathcal {N}}_{x_{0},D}$, and g(x), $x \in {\mathcal {N}}_{x_{0},D}$. Moreover, g(x) and $a(x_{0}) \in {\mathbb {R}}$ are Lagrange multipliers corresponding to inequality and equality constraints, respectively. Let $\omega ^{*}$ be the optimal solution to the minimization problem (13).

For the case $k=2$, a similar proof can be found in [53]. In this paper, we only demonstrate the case $k=1$. To find the solution $(\omega ^*, a(x_{0}), g)$ for the optimization problems mentioned above, the Karush-Kuhn-Tucker conditions must be satisfied. Specifically, the solution should satisfy the following equation, where

$$\begin{aligned} S(x_{0}) =\sqrt{\sum _{x\in {\mathcal {N}}_{x_{0},D}}{\omega ^{*}}^{2}(x)}, \end{aligned}$$

(A.1)

and we have the following equations:

$$\begin{aligned} \frac{{\partial {\mathcal {L}}}}{{\partial \omega (x)}}\bigg |_{\omega =\omega ^{*}}&= \phi (x,{x_0}) + \sqrt{\frac{2}{\pi }}\sigma \frac{{\omega ^{*}(x)}}{S(x_{0}) } - a(x_{0}) - g(x) = 0, \end{aligned}$$

(A.2)

$$\begin{aligned} \frac{{\partial {\mathcal {L}}}}{{\partial a(x_{0}) }}\bigg |_{\omega =\omega ^{*}}&= -\left( {\sum _{x \in {\mathcal {N}}_{x_{0},D}}\omega ^{*} (x) - 1} \right) = 0,\end{aligned}$$

(A.3)

$$\begin{aligned} \frac{{\partial {\mathcal {L}}}}{{\partial g(x)}}\bigg |_{\omega =\omega ^{*}}&= -\omega ^{*}(x)\left\{ \begin{array}{ll} = 0, & \quad if \ g(x) > 0,\\ \le 0, & \quad if \ g(x) = 0. \end{array} \right. \end{aligned}$$

(A.4)

First, we prove that the weight expression (14) holds, where $a(x_{0})$ satisfies (15). From (A.4), if $g(x)=0$, then $\omega ^{*}(x)\ge 0$, and substitute them into Eq. (A.2), we get

$$\begin{aligned} \omega ^{*}(x) = \frac{\sqrt{\frac{\pi }{2}}S(x_{0}) (a(x_{0})-\phi (x,x_{0}))}{\sigma } \ge 0, \end{aligned}$$

It is known that $\sigma > 0$ and $S(x_{0}) \ge 0$, so $a(x_{0})-\phi (x,x_{0})\ge 0$ in this case. If $g(x)>0$, we get $\omega ^{*}(x) = 0$ from (A.4). Similarly, substituting them into (A.2), we get

$$\begin{aligned} a(x_{0})-\phi (x,x_{0})=-g(x) < 0. \end{aligned}$$

Thus

$$\begin{aligned} \omega ^{*}(x)= \frac{\sqrt{\frac{\pi }{2}}S(x_{0}) \left[ a(x_{0})-\phi (x,x_{0}) \right] _{+}}{\sigma }. \end{aligned}$$

(A.5)

Then, from the equality constraint and expression (A.5), we know that

$$\begin{aligned} \sum _{x\in {\mathcal {N}}_{x_{0},D}}\frac{\sqrt{\frac{\pi }{2}}S(x_{0}) \left[ a(x_{0})-\phi (x,x_{0}) \right] _{+}}{\sigma } = 1, \end{aligned}$$

(A.6)

thereby $\sum _{x\in {\mathcal {N}}_{x_{0},D}}\sqrt{\frac{\pi }{2}}S(x_{0}) \left[ a(x_{0})-\phi (x,x_{0}) \right] _{+} = \sigma$, and substitute it into (A.6), we get

$$\begin{aligned} \omega ^{*}(x) = \frac{\left[ a(x_{0})-\phi (x,x_{0}) \right] _{+}}{\sum _{x\in {\mathcal {N}}_{x_{0},D}}\left[ a(x_{0})-\phi (x,x_{0}) \right] _{+}}. \end{aligned}$$

(A.7)

In addition, by substituting (A.5) into (A.3), we get

$$\begin{aligned} \left( \sum _{x\in {\mathcal {N}}_{x_{0},D}} \left[ a(x_{0})-\phi (x,x_{0}) \right] _{+} \right) ^{2}= \frac{2\sigma ^{2}}{\pi \sum _{x\in {\mathcal {N}}_{x_{0},D}}\omega ^{*}(x)^{2}}, \end{aligned}$$

and combining (A.7), we get the following equation

$$\begin{aligned} \sum _{x\in {\mathcal {N}}_{x_{0},D}}\left( \left[ a(x_{0})-\phi (x,x_{0}) \right] _{+} \right) ^{2} = \frac{2\sigma ^{2}}{\pi }. \end{aligned}$$

(A.8)

Therefore, the relation (14) is proved, and $a(x_{0})$ satisfies Eq. (A.8).

Next, we prove that $a(x_{0})$ is the unique solution of Eq. (A.8) and prove the uniqueness of the weights $\omega ^{*}(x)$. We define the function

$$\begin{aligned} {\mathcal {H}}(a(x_{0}))= \sum _{x\in {\mathcal {N}}_{x_{0},D}}\left( \left[ a(x_{0})-\phi (x,x_{0}) \right] _{+} \right) ^{2}, \end{aligned}$$

which is strictly increasing and continuous on $\left[ 0,+\infty \right)$. Further, we have ${\mathcal {H}}(0)= 0$ and $\lim \nolimits _{a(\! x_ {0}\! )\rightarrow +\! \infty }$ ${\mathcal {H}}(a(x_{0}))= +\infty$. So there exists a unique solution on $\left( 0,+\infty \right)$ of the equation ${\mathcal {H}}(a(x_{0}))= \frac{2\sigma ^{2}}{\pi }$. Moreover, according to the uniqueness of $a(x_{0})$ and (A.8) the weights $\omega ^* (x),\ x\in {{\mathcal {N}}_{x_{0},D}}$ are also uniquely determined by $\phi (x,x_{0}),$ $x\in {{\mathcal {N}}_{x_{0},D}}$.

1.2 The Proof of Theorem 2

Proof

The theorem with $k=2$ is similar to that in [53]. In this paper, we only demonstrate the theorem in the case $k=1$.

In order to solve the problem (17) with $k=1$, we introduce a series of auxiliary functions ${\mathcal {H}}_{j}(a(x_{0}))$, $j=0,1,\ldots ,M-1$ defined as follow

$$\begin{aligned} {\mathcal {H}}_{j}(a(x_{0}))= \sum _{i=0}^{j} \left( \left[ a(x_{0})- \phi (x_{i},x_{0}) \right] _{+} \right) ^{2}= \frac{2\sigma ^{2}}{\pi }. \end{aligned}$$

(A.9)

Let $a_{1,j}(x_{0})$, $j=0,1,\ldots ,M-1$ be the solution of ${\mathcal {H}}_{j}(a(x_{0}))$ respectively. Obviously, ${\mathcal {H}}_{M-1}(a(x_{0}))$ is equal to ${\mathcal {H}}(a(x_{0}))$. For convenience, denote $\phi (x_{M},x_{0})=+ \infty$.

Since function ${\mathcal {H}}(a(x_{0}))$ is strictly increasing on $\left( 0,+ \infty \right)$ with ${\mathcal {H}}(0)=0$ and ${\mathcal {H}}(+\infty )=+\infty$, Eq. (17) admits a unique solution $a(x_{0})$ on $\left( 0,+ \infty \right)$, which must be located in some interval $[ \phi (x_{{\overline{\jmath }}},x_{0}),$ $\phi (x_{{\overline{\jmath }}+1},x_{0}) )$, where $0\le {\overline{\jmath }} \le M-1$. Hence (17) with $k=1$ becomes

$$\begin{aligned} {\mathcal {H}}(a(x_{0}))=\sum _{i=0}^{{\overline{\jmath }}} \left( a(x_{0})- \phi (x_{i},x_{0}) \right) ^{2}= \frac{2\sigma ^{2}}{\pi }, \end{aligned}$$

where $\phi (x_{{\overline{\jmath }}},x_{0})\le a_{1,{\overline{\jmath }}}(x_{0})<\phi (x_{{\overline{\jmath }}+1},x_{0})$.

According to Eq. (A.9), it is easy to find that the function ${\mathcal {H}}_{j}(a(x_{0}))$ continuous and strictly increasing on $\left( 0,+ \infty \right)$ with ${\mathcal {H}}_{j}(0)=0$ and ${\mathcal {H}}_{j}(+ \infty )=+ \infty$. Therefore, there is an unique solution $a_{1,j}(x_{0})$ of (A.9) in $\left( 0,+ \infty \right)$ for each j. We first demonstrate that $a_{1,j}(x_{0})$ is decreasing monotonically with respect to j, i.e. $a_{1,j}(x_{0})\ge a_{1,j+1}(x_{0})$. $i=0,1,\ldots , M-1$. The hypothesis $a_{1,j+1} (x_{0})> a_{1,j}(x_{0}),j=0,1,\ldots ,M-1$ leads to

$$\begin{aligned} \begin{aligned} \sum \limits _{i=0}^{j+1} \! \left( \left[ a_{1,j+1}(x_{0})\! - \!\phi (x_{i},x_{0}) \right] _{+} \right) ^{2}&\ge \sum \limits _{i=0}^{j} \! \left( \left[ a_{1,j+1}(x_{0})\! - \!\phi (x_{i},x_{0}) \right] _{+} \right) ^{2} \\&= {a_{1,j+1}}^{2}(x_{0}) \! + \! \sum \limits _{i=1}^{j} \! \left( \left[ a_{1,j+1}(x_{0})\! - \!\phi (x_{i},x_{0}) \right] _{+} \right) ^{2} \\&> {a_{1,j}}^{2}(x_{0}) \! + \! \sum \limits _{i=1}^{j} \! \left( \left[ a_{1,j}(x_{0})\! - \!\phi (x_{i},x_{0}) \right] _{+} \right) ^{2} \\&= \sum \limits _{i=0}^{j} \! \left( \left[ a_{1,j}(x_{0})\! - \!\phi (x_{i},x_{0}) \right] _{+} \right) ^{2}\\&= \frac{2{\sigma }^{2}}{\pi }. \end{aligned} \end{aligned}$$

But $\sum \nolimits _{i=0}^{j+1} \! \left( \left[ a_{1,j+1}(x_{0})\! - \!\phi (x_{i},x_{0}) \right] _{+} \right) ^{2} =\frac{2{\sigma }^{2}}{\pi }$, so the hypothesis is not true, i.e.$a_{1,j}(x_{0})$ is decreasing monotonically with respect to j.

We then prove the theorem. When $j=0$, $a^2_{1,0}(x_{0}) = \frac{2{\sigma }^{2}}{\pi }$, i.e. $a_{1,0}(x_{0}) = \sqrt{\frac{2}{\pi }}\sigma >0=\phi (x_{0},x_{0})$. $a_{1,M}(x_{0})\le a_{1,0}(x_{0})<\phi (x_{M},x_{0})=+\infty$ due to $a_{1,j}(x_{0})$ is decreasing monotonically and with respect to j. Obviously, there exists $j^{*}$ that $j^{*}=\max \{j|\phi (x_{j},x_{0})\le a_{1,j}(x_{0})\}$, $0< j^{*} \le M-1$.

We now prove that ${\overline{\jmath }}=j^*$. It is sufficient to prove that $a_{1,j}(x_{0})< \phi (x_{j},x_{0})$ if ${\overline{\jmath }}<j\le M-1$. Since $a_{1,j}(x_{0})$ is decreasing monotonically and $\phi (x_{j},x_{0})$ is monotonically increasing respect to j, for $j>{\overline{\jmath }}$ we have

$$\begin{aligned} a_{1,j}(x_{0})\le a_{1,{\overline{\jmath }}}(x_{0}) <\phi (x_{{\overline{\jmath }}+1},x_{0})\le \phi (x_{j},x_{0}). \end{aligned}$$

(A.10)

We finally show that if $0\le j < M-1$ and $a_{1,j}(x_{0})<\phi (x_{j},x_{0})$, so that $j^* = \max \{j|a_{1,j}(x_{0})\ge \phi (x_{j},x_{0})\}$ is the unique integer $j\in \{1,2,\cdots ,M-1\}$ such that $a_{1,j}(x_{0})\ge \phi (x_{j},x_{0})$ and $a_{1,j+1}(x_{0})< \phi (x_{j+1},x_{0})$ if $0\le j<M-1$. According to $a_{1,j}(x_{0})$ is decreasing monotonically and $\phi (x_{j},x_{0})$ is monotonically increasing respect to j, the inequality $a_{1,j}(x_{0})< \phi (x_{j},x_{0})$ leads to

$$\begin{aligned} a_{1,j+1} (x_{0})\le a_{1,j}(x_{0})< \phi (x_{j},x_{0}) \le \phi (x_{j+1},x_{0}). \end{aligned}$$

$\square$

1.3 The Proof of Theorem 3

We first prove the Eq. (28) holds for $x_{0}\in {\Omega }$. The objective function ${\mathcal {J}}_{k,\phi }(\omega )$ is known to be defined by (12) and the optimal weights $\omega$ is defined by (14), where $\phi (x,x_{0}) = \left| u(x)-u(x_{0}) \right|$. According to the Hölder condition (26) we know that for any $\omega$, we get

$$\begin{aligned} {\mathcal {J}}_{k,\phi }(\omega )\le \tilde{{\mathcal {J}}}_{k,\phi }(\omega ), \end{aligned}$$

(A.11)

where

$$\begin{aligned} \tilde{{\mathcal {J}}}_{k,\phi }(\omega )=\left( \sum _{x\in {\mathcal {N}}_{x_{0},D}}\omega (x)L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) ^{k}+ \theta {\sigma ^k}\left( {\sum _{x\in {\mathcal {N}}_{x_{0},D}}\omega ^{2} (x)}\right) ^{\frac{k}{2}}, \end{aligned}$$

(A.12)

where $\theta =\sqrt{\frac{2}{\pi }}$ when $k=1$, and $\theta =1$ when $k=2$. From Theorem 1,

$$\begin{aligned} {\tilde{\omega }}(x)=\frac{\left[ {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+}}{\sum _{x\in {\mathcal {N}}_{x_{0},D}}\left[ {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+}}, \end{aligned}$$

(A.13)

where ${\tilde{a}}(x_{0})> 0$ is the unique solution on $\left( 0, +\infty \right)$ to the equation

$$\begin{aligned} \tilde{{\mathcal {H}}}_{h}(z):= \left\{ \begin{aligned}&\sum _{x\in {\mathcal {N}}_{x_{0},D}} \left( \left[ z - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+}\right) ^{2}= \frac{2\sigma ^{2}}{\pi }, \quad k=1;\\&\sum _{x\in {\mathcal {N}}_{x_{0},D}} {L}\left\| x-x_{0} \right\| ^{\beta }_{\infty } \left[ z - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+} =\sigma ^{2}, \quad k=2, \end{aligned} \right. \end{aligned}$$

(A.14)

where $z\ge 0$.

Lemma 1

Suppose that $c_{1} n^{-\alpha }\le h\le 1$, where $0\le \alpha \le \frac{1}{2\beta +2}$ and $c_{1}> 0$ if $0 \le \alpha < \frac{1}{2\beta +2}$, $c_{1}> c_{0} = \left( \frac{\sigma ^{2}(2\beta +2)(\beta + 2)}{R L^{2}} \right) ^{\frac{1}{2\beta + 2}}$ if $\alpha = \frac{1}{2\beta +2}$. Then

$$\begin{aligned} {\tilde{a}}(x_{0})=c_{2} n^{-\frac{\beta }{2\beta +2}}(1+o(1)), \end{aligned}$$

(A.15)

and

$$\begin{aligned} \tilde{{\mathcal {J}}}_{k,\phi }({\tilde{\omega }})\le c_{3} n^{-\frac{k\beta }{2\beta +2}}, \end{aligned}$$

(A.16)

where $k=1,2$. $R=4\pi \beta ^2$ when $k=1$, and $R=8\beta$ when $k=2$. o(1) denotes an infinite small quantity and $c_{2}$, $c_{3}$ are positive constants that depend only on $\beta$, L and $\sigma$.

Proof

We first prove that (A.15) holds when $h = 1$ (that is, ${\mathcal {N}}_{x_{0},D} = {\Omega }$). According to the definition of ${\tilde{a}}(x_{0})$ (see (A.14)), we have

$$\begin{aligned} \tilde{{\mathcal {H}}}_{1}({\tilde{a}}(x_{0})) =\left\{ \begin{aligned}&\sum _{x\in {\Omega } }\left( \left[ {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+} \right) ^{2}= \frac{2\sigma ^{2}}{\pi },\quad k=1;\\&\sum _{x\in {\Omega } } L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\left[ {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+}=\sigma ^{2},\quad k=2, \end{aligned} \right. \end{aligned}$$

(A.17)

Let ${\tilde{h}}=\left( \frac{{\tilde{a}}(x_{0})}{L} \right) ^{\frac{1}{\beta }}$, then ${\tilde{a}}(x_{0})= L{\tilde{h}}^{\beta }$, where by we get

$$\begin{aligned} \left[ {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\right] _{+}= 0,\ \text {iff} \ \left\| x-x_{0} \right\| _{\infty }> {\tilde{h}}. \end{aligned}$$

(A.18)

According to (A.18) and ${\tilde{a}}(x_{0})= L{\tilde{h}}^{\beta }$, (A.17) becomes

$$\begin{aligned} \tilde{{\mathcal {H}}}_{{\tilde{h}}}({\tilde{a}}(x_{0})):=\left\{ \begin{aligned}&L^{2} \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{h}}^{\beta } - \left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) ^{2} = \frac{2\sigma ^{2}}{\pi },\quad \quad \quad \quad \quad \quad k=1;\\&L^{2} \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{h}}^{\beta } - \left\| x-x_{0} \right\| ^{\beta }_{\infty } - \left\| x-x_{0} \right\| ^{2\beta }_{\infty } \right) =\sigma ^{2},\quad k=2. \end{aligned} \right. \end{aligned}$$

(A.19)

By the definition of the neighbourhood ${\mathcal {N}}_{x_{0},{\tilde{D}}}$ with ${\tilde{h}}=\frac{{\tilde{D}}-1}{2N}$, we easily know that

$$\begin{aligned} \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left\| x-x_{0} \right\| ^{\beta }_{\infty } = 8\sum _{i=1}^{N{\tilde{h}}}i\left( \frac{i}{N}\right) ^{\beta } = \frac{8N^{2}{\tilde{h}}^{\beta +2}}{\beta +2}(1+o(1)), \end{aligned}$$

(A.20)

where o(1) denotes an infinitely small quantity, this formula also applies to the case where $2\beta$ is used instead of $\beta$, so

$$\begin{aligned} \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left\| x-x_{0} \right\| ^{2 \beta }_{\infty }= \frac{8N^{2}{\tilde{h}}^{2\beta +2}}{2 \beta +2}(1+o(1)). \end{aligned}$$

(A.21)

Thus, (A.19) implies

$$\begin{aligned} \tilde{{\mathcal {H}}}_{{\tilde{h}}}({\tilde{a}}(x_{0}))=\left\{ \begin{aligned}&\frac{8L^{2}\beta ^{2}}{(2 \beta +2)(\beta +2)}N^{2}{\tilde{h}}^{2\beta +2}\left( 1+o(1) \right) = \frac{2\sigma ^{2}}{\pi },\quad k=1;\\&\frac{8L^{2}\beta }{(2 \beta +2)(\beta +2)}N^{2}{\tilde{h}}^{2\beta +2}\left( 1+o(1) \right) = \sigma ^{2},\quad k=2, \end{aligned} \right. \end{aligned}$$

(A.22)

from which we infer that

$$\begin{aligned} {\tilde{h}}= c_{0}n^{-\frac{1}{2\beta +2}}(1+o(1)), \end{aligned}$$

(A.23)

where

$$\begin{aligned} c_{0}= \left\{ \begin{aligned}&\left( \frac{\sigma ^{2}(2\beta +2)(\beta +2)}{4\pi L^{2}\beta ^{2}} \right) ^{\frac{1}{2\beta +2}},\quad k=1;\\&\left( \frac{\sigma ^{2}(2\beta +2)(\beta +2)}{8 L^{2}\beta } \right) ^{\frac{1}{2\beta +2}},\quad k=2. \end{aligned} \right. \end{aligned}$$

By (A.23) and the definition of ${\tilde{h}}$, we get

$$\begin{aligned} {\tilde{a}}(x_{0})=L{\tilde{h}}^{\beta }= L c_{0}^{\beta }n^{-\frac{\beta }{2\beta +2}}(1+o(1))= c_{2}n^{-\frac{\beta }{2\beta +2}}(1+o(1)), \end{aligned}$$

which proves (A.15) in the case when $h = 1$.

We now prove (A.15) for $c_{1} n^{-\alpha }\le h< 1$, where $0\le \alpha \le \frac{1}{2\beta +2}$ and $c_{1}> 0$ if $0 \le \alpha < \frac{1}{2\beta +2}$, $c_{1}> c_{0}$ if $\alpha = \frac{1}{2\beta +2}$. It is easy to know that $h\ge {\tilde{h}}= c_{0}n^{-\frac{1 }{2\beta +2}}(1+o(1))$ for n to be large enough. Using again (A.18), we see that

$$\begin{aligned} \tilde{{\mathcal {H}}}_{1}({\tilde{a}}(x_{0}))= \tilde{{\mathcal {H}}}_{h}({\tilde{a}}(x_{0}))= \tilde{{\mathcal {H}}}_{{\tilde{h}}}({\tilde{a}}(x_{0})), \end{aligned}$$

from this formula it follows that if h satisfies $c_{1}n^{-\alpha }\le h <1$, then ${\tilde{a}}(x_{0})= c_{2}n^{-\frac{\beta }{2\beta +2}}(1+o(1))$ is also the solution of the Eq. (A.14). This completes the proof of (A.15).

The case $k=2$ is the same to [53], in this paper, we only prove the case $k=1$.

By substituting (A.13) in (A.12) and using the equivalence (A.19), we get

$$\begin{aligned} \tilde{{\mathcal {J}}}_{k,\phi }({\tilde{\omega }})&= \frac{\sqrt{\pi } \sum \limits _{\left\| x-x_{0} \right\| _{\infty }\le h}\left[ {\tilde{a}}(x_{0})\! - L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \! \right] _{+}\! L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\! + \! \sigma \sqrt{\! 2\tilde{{\mathcal {H}}}_{h}({\tilde{a}}(\! x_{0}\! ))}}{\sqrt{\pi }\sum \limits _{\left\| x-x_{0} \right\| _{\infty }\le h}\left[ {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \right] _{+}} \nonumber \\&=\frac{\sqrt{\pi } \sum \limits _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{a}}(x_{0})\! - L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \! \right) \! L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\! + \! \sigma \sqrt{\! 2\tilde{{\mathcal {H}}}_{{\tilde{h}}}({\tilde{a}}(\! x_{0}\! ))}}{\sqrt{\pi }\sum \limits _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) } \nonumber \\&=\frac{\pi \sum \limits _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{a}}(x_{0})\! - L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \! \right) \! L\left\| x-x_{0} \right\| ^{\beta }_{\infty }\! + 2\sigma ^{2}}{\pi \sum \limits _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{a}}(x_{0}) - L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) } \nonumber \\&=\frac{\pi L^{2}P_{{\tilde{h}}}+ 2\sigma ^{2}}{\pi LQ_{{\tilde{h}}}}, \end{aligned}$$

(A.24)

where

$$\begin{aligned} P_{{\tilde{h}}}= & \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{h}}^{\beta } - \left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) \left\| x-x_{0} \right\| ^{\beta }_{\infty },\\ Q_{{\tilde{h}}}= & \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\left( {\tilde{h}}^{\beta } - \left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) , \end{aligned}$$

and the last equality holds from the fact that ${\tilde{a}}(x_{0})=L{\tilde{h}}^{\beta }$. Since

$$\begin{aligned} \begin{aligned} -L^{2}P_{{\tilde{h}}}&= \! \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\! \left( L{\tilde{h}}^{\beta }\! - \! L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) ^{2}+ L^{2}{\tilde{h}}^{\beta }\! \! \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\! \left( \left\| x-x_{0} \right\| ^{\beta }_{\infty }\! - \! {\tilde{h}}^{\beta } \right) \\&=\! \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\! \left( {\tilde{a}}(\! x_{0}\! )\! - \! L\left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) ^{2}\! - \! L^{2}{\tilde{h}}^{\beta }\! \! \sum _{\left\| x-x_{0} \right\| _{\infty }\le {\tilde{h}}}\! \left( {\tilde{h}}^{\beta }\! - \! \left\| x-x_{0} \right\| ^{\beta }_{\infty } \right) \\&=\frac{2\sigma ^{2}}{\pi }-L^{2}{\tilde{h}}^{\beta }Q_{{\tilde{h}}}, \end{aligned} \end{aligned}$$

where the last equality is obtained by using (A.19), we have $\pi L^{2}P_{{\tilde{h}}}= \pi L^{2}{\tilde{h}}^{\beta }Q_{{\tilde{h}}}- 2\sigma ^{2}$. Substituting to (A.24) and using (A.23), we obtain

$$\begin{aligned} \tilde{{\mathcal {J}}}_{k,\phi }({\tilde{\omega }})= L{\tilde{h}}^{\beta }= Lc_{0}^{\beta }n^{-\frac{\beta }{2\beta +2}}\left( 1+o(1) \right) = c_{3}n^{-\frac{\beta }{2\beta +2}}\left( 1+o(1) \right) , \end{aligned}$$

where $c_{3}=Lc_0^\beta$ and depends on $\beta$, L, and $\sigma$. $\square$

See from (11), because $\sum _{x\in {\mathcal {N}}_{x_{0},D}}\omega (x)\varepsilon (x) \sim N(0, {\sigma }^2\sum _{x\in {\mathcal {N}}_{x_{0},D}} {\omega ^2(x)})$ when $D\rightarrow +\infty$, we have $\delta ^{E}_{n}= O(n^{-\frac{\beta }{2\beta +2}})$ for sufficiently large D. It is easy to get

$$\begin{aligned} {\mathbb {E}}\left| {\hat{u}}(x_{0})-u(x_{0}) \right| \le {\mathcal {J}}_{1,\phi }(\omega ) + O(n^{-\frac{\beta }{2\beta +2}}). \end{aligned}$$

Hence, Lemma 1, (A.11) and (12) indicate that the Eq. (28) holds for $k=1$ and $x_{0}\in {\Omega }$, while Lemma 1, (A.11) and (10) imply that the Eq. (28) holds for $k=2$ and $x_{0}\in {\Omega }$.

We finally prove that the Eq. (28) holds for all the points on ${\Omega }_{0}$.

Let $x_{0}\in {\Omega }_{0}$, we have $x'_{0} = (x'_{0,1},x'_{0,2})= \left( \frac{[N x_{0,1}]}{N},\frac{[N x_{0,2}]}{N}\right) \in {\Omega }$ and ${\hat{u}}_{k,D}^{*}(x_{0}) = {\hat{u}}_{k,D}^{*}(x'_{0})$, $k=1$ and 2. The local Hölder condition (26) implies that

$$\begin{aligned} {\mathbb {E}}\left| {\hat{u}}_{k,D}^{*}(x_{0})-u(x_{0}) \right| ^{k}= & {\mathbb {E}}\left| {\hat{u}}_{k,D}^{*}(x'_{0})-u(x_{0}) \right| ^{k}\\= & {\mathbb {E}}\left| {\hat{u}}_{k,D}^{*}(x'_{0})-u(x'_{0})+u(x'_{0})-u(x_{0}) \right| ^{k}\\\le & k({\mathbb {E}}\left| {\hat{u}}_{k,D}^{*}(x'_{0})-u(x'_{0}) \right| ^{k}+ \left| u(x'_{0})-u(x_{0}) \right| ^{k})\\= & k(O(n^{-\frac{k\beta }{2\beta + 2}}))+ k\left( L(\sqrt{2}n^{-2})^{\beta }\right) ^{k}\\= & O(n^{-\frac{k\beta }{2\beta + 2}}). \end{aligned}$$

For inequalities, when $k=1$, the absolute value inequality clearly holds. When $k=2$, it can be obtained from the Cauchy-Schwartz inequality. Then for all point $x_{0}\in {\Omega }_{0}$, the Eq. (28) holds.

1.4 The Proof of Theorem 4

From the condition $\phi (x,x_{0})\le \alpha \left| u(x)- u(x_{0}) \right| + \varepsilon _{n}$, where $\alpha > 0$ is a constant, and $\varepsilon _{n}(x)= O(n^{-\frac{\beta }{2\beta + 2}})$, we know that

$$\begin{aligned} \begin{aligned} \left( \sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\phi (x,x_{0}) \right) ^{k}\!\le \! \left( \sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\left( \alpha \left| u(x)- u(x_{0}) \right| + \varepsilon _{n} \right) \right) ^{k}, \end{aligned} \end{aligned}$$

where $k=1,2$. In the case of $k=1$,

$$\begin{aligned} \begin{aligned} \sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\left( \alpha \left| u(x)- u(x_{0}) \right| + \varepsilon _{n} \right)&= \alpha \!\sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\left| u(x)- u(x_{0}) \right| + \sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\varepsilon _{n}\\&\le \alpha \!\sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\left| u(x)- u(x_{0}) \right| + O(n^{-\frac{\beta }{2\beta + 2}}). \end{aligned} \end{aligned}$$

In the case of $k=2$, as $(a+b)^2 \le 2a^2+2b^2$, the solution of the above equation is

$$\begin{aligned} \begin{aligned} \left( \sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\left( \alpha \left| u(x)- u(x_{0}) \right| + \varepsilon _{n} \right) \right) ^{2}&\le 2\alpha ^{2}\left( \sum _{x\in {{\mathcal {N}}_{x_{0},D}}}\omega (x)\left| u(x)- u(x_{0}) \right| \right) ^{2}\\&\quad + O(n^{-\frac{2\beta }{2\beta + 2}}). \end{aligned} \end{aligned}$$

Hence

$$\begin{aligned} {\mathcal {J}}_{k,\phi }(\omega )\le \max \left\{ \alpha ,2\alpha ^{2}\right\} \tilde{{\mathcal {J}}}_{k,\phi }(\omega )+O(n^{-\frac{k\beta }{2\beta + 2}}). \end{aligned}$$

Setting ${\tilde{\omega }}= \mathop {\arg \min }_{\omega }\tilde{{\mathcal {J}}}_{k,\phi }(\omega )$ and using (10) and (11), we have

$$\begin{aligned} {\mathbb {E}}\left| {\hat{u}}_{k,D}^{*}(x_{0})- u(x_{0}) \right| ^{k} \! \le \! \min _{\omega }{\mathcal {J}}_{k,\phi }(\omega )\! \le \! {\mathcal {J}}_{k,\phi }({\tilde{\omega }}) \! \le \!\max \left\{ \alpha ,2\alpha ^{2}\right\} \tilde{{\mathcal {J}}}_{k,\phi }({\tilde{\omega }})+O(n^{-\frac{k\beta }{2\beta + 2}}). \end{aligned}$$

So, from Lemma 1, we get

$$\begin{aligned} {\mathbb {E}}\left| {\hat{u}}_{k,D}^{*}(x_{0})- u(x_{0}) \right| ^{k}= O\left( n^{-\frac{{k}\beta }{2\beta +2}} \right) . \end{aligned}$$

where $k=1$ or 2.

From the proof of Theorem 3, it is easy to prove that the Eq. (30) holds for all the points on ${\Omega }_{0}$.

1.5 The Proof of Theorem 5

The case $k=2$ is similar to [53], in this paper, we only prove the case $k=1$. For simplicity, we shall assume that the kernel $\kappa$ is rectangular; the proof in the general case can be carried out in the same way. Note that for the rectangular kernel we have $\left\| v({\mathcal {N}}_{x,d}) - v({\mathcal {N}}_{x_{0},d}) \right\| _{1,\kappa } =\frac{1}{m}\left\| v({\mathcal {N}}_{x,d}) - v({\mathcal {N}}_{x_{0},d}) \right\| _{1}$, where m is the number of points in the similarity patch. Under the Hölder condition (26), for $x\in {\mathcal {N}}_{x_{0},D}$ we have

$$\begin{aligned} & \frac{1}{m}\left\| v({\mathcal {N}}_{x,d}) - v({\mathcal {N}}_{x_{0},d}) \right\| _{1} \\ & \quad = \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |u(x+t)+\varepsilon _{n}(x+t)-u(x_{0}+t)-\varepsilon _{n}(x_{0}+t)|\\ & \quad \le \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |u(x+t)-u(x_{0}+t)|+\frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |\varepsilon _{n}(x+t)-\varepsilon _{n}(x_{0}+t)|\\ & \quad \le \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |u(x+t)+u(x)-u(x)-u(x_{0}+t)-u(x_{0})+u(x_{0})|\\ & \qquad +\frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |\varepsilon _{n}(x+t)-\varepsilon _{n}(x_{0}+t)|\\ & \quad \le \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |u(x)-u(x_{0})|+ \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |u(x+t)-u(x)|\\ & \qquad + \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |u(x_{0}+t)-u(x_{0})|+\frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |\varepsilon _{n}(x+t)-\varepsilon _{n}(x_{0}+t)|\\ & \quad \le |u(x)-u(x_{0})| + 2L\eta ^{\beta } +\frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |\varepsilon _{n}(x+t)-\varepsilon _{n}(x_{0}+t)|. \end{aligned}$$

Considering ${\mathbb {E}}\left| \varepsilon (x)-\varepsilon (x_{0}) \right| =\mu$, we get

$$\begin{aligned} & P\left\{ \max _{0\le j \le m} \left| \frac{1}{m}\sum _{t \in {\mathcal {N}}_{0,d} } |\varepsilon _{n}(x+t)-\varepsilon _{n}(x_{0}+t)| -\mu \right| \ge n^{-\frac{\beta }{2+2\beta }} \right\} \\ & \quad \le \frac{n^{\frac{2\beta }{2+2\beta }}}{m} Var\left| \varepsilon (x)-\varepsilon (x_{0}) \right| \\ & \quad = n^{2(\alpha -\frac{1}{2\beta +2})}(2\sigma ^{2}-\mu ^2). \end{aligned}$$

Because $\frac{1}{2(\beta +1)^{2}}<\alpha <\frac{1}{2 \beta +2}$, $n^{2(\alpha -\frac{1}{2\beta +2})}\rightarrow 0$ with $n\rightarrow \infty$. Hence with high probability, we have

$$\begin{aligned} \frac{1}{m}\left\| v({\mathcal {N}}_{x,d}) - v({\mathcal {N}}_{x_{0},d}) \right\| _{1} \le |u(x)-u(x_{0})|+2L\eta ^{\beta } + \mu + n^{-\frac{\beta }{2\beta +2}}. \end{aligned}$$

(A.25)

Taking into account that for $\alpha$ satisfying $\frac{1}{4 \beta +2} \le \alpha \le \frac{1}{2 \beta +2}$, it holds that $2 \alpha \beta \ge 1 / 2-\alpha$, we have $2\,L\eta ^{\beta }\le O(n^{-2\alpha \beta })\le O(n^{\alpha -\frac{1}{2}})=O(n^{-\frac{\beta }{2\beta +2}})$. Then with high probability, Eq. (A.25) becomes

$$\begin{aligned} \frac{1}{m}\left\| v({\mathcal {N}}_{x,d}) - v({\mathcal {N}}_{x_{0},d}) \right\| _{1} \le |u(x)-u(x_{0})|+ \mu + O(n^{-\frac{\beta }{2\beta +2}}). \end{aligned}$$

(A.26)

This implies that with a high probability

$$\begin{aligned} {\hat{\phi }}_{1}(x,x_{0})\le |u(x)-u(x_{0})|+ O(n^{-\frac{\beta }{2\beta +2}}). \end{aligned}$$

(A.27)