Robust Diffusion Huber-Based Normalized Least Mean Square Algorithm with Adjustable Thresholds

Abstract

To improve the performance of the diffusion Huber-based normalized least mean square algorithm in the presence of impulsive noise, this paper proposes a distributed recursion scheme to adjust the thresholds. Because of the decreasing characteristic of the thresholds, the proposed algorithm can also be interpreted as a robust diffusion normalized least mean square algorithm with variable step sizes so that it has not only fast convergence but also small steady-state estimation error. Based on the contaminated Gaussian model, we analyze the mean square behavior of the algorithm in impulsive noise. Moreover, to ensure good tracking capability of the algorithm for the sudden change of parameters of interest, a control strategy is given that resets the thresholds with their initial values. Simulations in various noise scenarios show that the proposed algorithm performs better than many existing diffusion algorithms.

Proof of \(E\{\xi _{k}(i)\}\) Approximately Converging to Zero

We define the threshold vector over the network and its intermediate counterpart, as follows:

$$\begin{aligned} \begin{aligned} {\varvec{\xi }}(i)&\triangleq \text {col}\{\xi _1(i), \ldots , \xi _N(i)\}, \\ {\varvec{\zeta }}(i)&\triangleq \text {col}\{\zeta _1(i), \ldots , \zeta _N(i)\}. \\ \end{aligned} \end{aligned}$$

(A.1)

Following this, equations (12) and (13) for all the nodes can be expressed in a matrix form:

$$\begin{aligned} \begin{aligned} {\varvec{\xi }} (i+1)&={\varvec{C}}^{\mathrm{T}} {\varvec{\zeta }} (i+1)\\&= {\varvec{C}}^{\mathrm{T}} \left[ \upsilon {\varvec{\xi }}(i) + (1-\upsilon ) \min [{\varvec{f}}(i),{\varvec{\xi }}(i)] \right] , \end{aligned} \end{aligned}$$

(A.2)

where

$$\begin{aligned} {\varvec{f}}(i) \triangleq \text {col}\left\{ \frac{e_1^2(i)}{\Vert {\varvec{u}}_1(i)\Vert _2^2}, \ldots , \frac{e_N^2(i)}{\Vert {\varvec{u}}_N(i)\Vert _2^2} \right\} . \end{aligned}$$

(A.3)

Taking the $\infty$-norm both sides of (A.2), we get

$$\begin{aligned} \begin{aligned} \Vert {\varvec{\xi }} (i+1)\Vert _\infty&\le \Vert {\varvec{C}}^{\mathrm{T}} \Vert _\infty \cdot \Vert {\varvec{\zeta }} (i+1)\Vert _\infty \\&= \Vert {\varvec{\zeta }} (i+1)\Vert _\infty \\&\le \Vert \upsilon {\varvec{\xi }}(i) + (1-\upsilon ) \min \left[ {\varvec{f}}(i), {\varvec{\xi }}(i) \right] \Vert _\infty \\&\le \Vert {\varvec{\xi }}(i)\Vert _\infty . \end{aligned} \end{aligned}$$

(A.4)

Since $\{\xi _k(i)\}$ is a positive sequence, (A.4) can deduce that $\xi _k(i)$ and $\zeta _k(i)$ from (12) and (13) are decreasing as the iteration i increases. It follows that the expectations $E\{\xi _k(i)\}$ and $E\{\zeta _k(i)\}$ are also positive and decreasing. Thus, taking the expectations of both sides of (A.2), we have

$$\begin{aligned} \begin{aligned} E\{{\varvec{\xi }} (i+1)\} = {\varvec{C}}^{\mathrm{T}} [ \upsilon E\{{\varvec{\xi }}(i)\} +(1-\upsilon ) E\{\min [{\varvec{f}}(i),{\varvec{\xi }}(i)]\} ]. \end{aligned} \end{aligned}$$

(A.5)

To continue developing the expression (A.5), again using the assumption that the variance of $\xi _k(i)$ is small enough so that we can make the following approximation,

$$\begin{aligned} \begin{aligned} E\left\{ \min \left[ \frac{e_k^2(i)}{\Vert {\varvec{u}}_k(i)\Vert _2^2},\xi _k(i) \right] \right\}&\approx \\ E\{\xi _k(i)\}P_{k,i}[z_k>&E\{\xi _k(i)\}] + \int _{0}^{E\{\xi _k(i)\}}z_k \mathrm{d}F_{k,i}(z_k), \end{aligned} \end{aligned}$$

(A.6)

where $z_k\doteq e_k^2(i)/ \Vert {\varvec{u}}_k(i)\Vert _2^2$ denotes that both $z_k$ and $e_k^2(i)/ \Vert {\varvec{u}}_k(i)\Vert _2^2$ have the same distribution, $P_{k,i}[a]$ denotes the probability of event a, and $F_{k,i}(z_k)$ denotes the distribution function of $z_k$ at time instant i at node k. Plugging (A.6) into (A.5), we have

$$\begin{aligned} E\{{\varvec{\xi }} (i+1)\} = {\varvec{C}}^{\mathrm{T}} [ \upsilon E\{{\varvec{\xi }}(i)\} + (1-\upsilon ) {\varvec{P}}(i)E\{{\varvec{\xi }}(i)\}+(1-\upsilon ){\varvec{z}}(i)], \end{aligned}$$

(A.7)

where

$$\begin{aligned} {\varvec{P}}(i) = \text {diag}\left\{ P_{1,i}[z_1>E\{\xi _1(i)\}], \ldots , P_{N,i}[z_N>E\{\xi _N(i)\}] \right\} , \end{aligned}$$

(A.8)

and

$$\begin{aligned} {\varvec{z}}(i) = \text {col}\left\{ \int _{0}^{E\{\xi _1(i)\}}z_1 \mathrm{d}F_{1,i}(z_1), \ldots , \int _{0}^{E\{\xi _N(i)\}}z_N \mathrm{d}F_{N,i}(z_N) \right\} . \end{aligned}$$

(A.9)

Taking the limits for both sides of (A.7) as $i\rightarrow \infty$, we obtain

$$\begin{aligned} E\{{\varvec{\xi }} (\infty )\} = {\varvec{C}}^{\mathrm{T}} [ \upsilon E\{{\varvec{\xi }}(\infty )\}+(1-\upsilon ) {\varvec{P}}(\infty )E\{{\varvec{\xi }}(\infty )\}+(1-\upsilon ){\varvec{z}}(\infty )]. \end{aligned}$$

(A.10)

Then, we take the $\infty$-norm of both sides of (A.10), and after some simple manipulations with $\Vert {\varvec{C}}^{\mathrm{T}}\Vert _\infty =1$, getting the following inequality:

$$\begin{aligned} \Vert E\{{\varvec{\xi }} (\infty )\}\Vert _\infty \le \Vert {\varvec{P}}(\infty )\Vert _\infty \cdot \Vert E\{{\varvec{\xi }}(\infty )\}\Vert _\infty +\Vert {\varvec{z}}(\infty )\Vert _\infty , \end{aligned}$$

(A.11)

where

$$\begin{aligned} \Vert {\varvec{P}}(\infty )\Vert _\infty = \max _{1 \le k \le N} P_{k,\infty }[z_k>E\{\xi _k(\infty )\}]. \end{aligned}$$

(A.12)

Due to the property of (A.12), we can equivalently formulate (A.11) as

$$\begin{aligned} \begin{aligned} E\{\xi _k (\infty )\}&\le P_{k,\infty }[z_k>E\{\xi _k(\infty )\}] \cdot E\{\xi _k(\infty )\}\\&\quad +\,\int _{0}^{E\{\xi _k(\infty )\}}z_k \mathrm{d}F_{k,\infty }(z_k), \end{aligned} \end{aligned}$$

(A.13)

for $k=1,\ldots ,N$, which further results in

$$\begin{aligned} \begin{aligned} P_{k,\infty }[z_k \le E\{\xi _k(\infty )\}] \cdot E\{\xi _k(\infty )\} \le \int _{0}^{E\{\xi _k(\infty )\}}z_k \mathrm{d}F_{k,\infty }(z_k). \end{aligned} \end{aligned}$$

(A.14)

It is difficult to solve (A.14) with respect to $E\{\xi _k(\infty )\}$, and thus for mathematical tractability, we consider solving the equal sign case in (A.14), i.e.,

$$\begin{aligned} P_{k,\infty }[z_k \le E\{\xi _k(\infty )\}] \cdot E\{\xi _k(\infty )\} = \int _{0}^{E\{\xi _k(\infty )\}}z_k \mathrm{d}F_{k,\infty }(z_k). \end{aligned}$$

(A.15)

For Eq. (A.15), Appendix A in [48] can be applied to obtain its solution that is $E\{\xi _k(\infty )\}]=0$. Because (A.15) is the upper bound of (A.14), we can infer that $E\{\xi _k(i)\}$ for nodes $k=1,\ldots ,N$ will approximately converge to zero. Likewise, the intermediate value $E\{\zeta _k(i)\}$ per node k also approximately converges to zero.