Clipped LMS/RLS Adaptive Algorithms: Analytical Evaluation and Performance Comparison with Low-Complexity Counterparts

Abstract

The high computational load of conventional adaptive FIR filters applied to long system identification and their weak tracking ability has encouraged researchers to seek for efficient adaptive algorithms for this kind of applications. One of the efficient solutions is the three-level clipped LMS/RLS adaptive algorithm. In this paper, an insight into the performance of these adaptive algorithms that evaluates the amount of steady-state misalignment error of clipped LMS/RLS adaptive algorithms employed for identification of time-invariant and time-varying systems, is presented. Employing it, we compare the misalignment performance with their low-complexity adaptive algorithm counterparts theoretically. In addition, we derive the optimal step size/forgetting factor explicitly and also obtain a relation between the optimal level of the clipping and step size/forgetting factor to achieve the lowest steady-state misalignment and then we explain as to how to improve the performance of these kinds of adaptive algorithms by adjusting the clipping threshold according to the noise level in such a way that a higher performance is achieved. Finally, different adaptive algorithms have been further coded in VHDL in order to evaluate them in terms of speed and hardware resources.

Appendices

Appendix 1

To derive $\eta$, we utilize (3) and (5) giving

$$\begin{aligned} \mathbf {v}(n+1)&=\mathbf {\widehat{h}}(n+1)-\mathbf {h}\nonumber \\&=\Big \{\mathbf {\widehat{h}}(n)+\varvec{\Psi }(n) e(n)\mathbf {\widetilde{x}}(n)\Big \}-\mathbf {h}\nonumber \\&=\Big \{\mathbf {\widehat{h}}(n)+\varvec{\Psi }(n) \mathbf {\widetilde{x}}(n)[d(n)-\mathbf {x}^\mathrm{{T}}(n)\mathbf {\widehat{h}}(n)]\Big \}-\mathbf {h}. \end{aligned}$$

(32)

Making $w(n)$ the subject of (1) and substituting the resultant equation to the above, we obtain

$$\begin{aligned} \mathbf {v}(n+1)&=\mathbf {\widehat{h}}(n)-\mathbf {h}+\varvec{\Psi }(n)\mathbf {\widetilde{x}}(n) \big \{w(n)-\mathbf {x}^\mathrm{{T}}(n)\big [\mathbf {\widehat{h}}(n)-\mathbf {h}\big ]\big \}\nonumber \\&=\mathbf {v}(n)+\varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)\big \{w(n)-\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\big \}\nonumber \\&=\mathbf {v}(n)+\varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)w(n)-\varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n) \end{aligned}$$

(33)

and hence,

$$\begin{aligned} \mathbf {Q}(n+1)&= \mathbf {Q}(n)+ E\left\{ \varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)w^2(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\varvec{\Psi }^\mathrm{{T}}(n)\right\} \nonumber \\&\quad - E\left\{ \mathbf {v}(n)\big [\varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\big ]^\mathrm{{T}}\right\} \nonumber \\&\quad - E\left\{ \big [\varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\big ]\mathbf {v}^\mathrm{{T}}(n)\right\} \nonumber \\&\quad + E\left\{ \varvec{\Psi }(n)\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\varvec{\Psi }^\mathrm{{T}}(n)\right\} . \end{aligned}$$

(34)

To simplify the above expression, we assume that $w(n)$ is uncorrelated with $x(n)$. Denoting $\mathbf {C}(n)=E\{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\}$, we can express (34) as

$$\begin{aligned} \mathbf {Q}(n+1)&= \mathbf {Q}(n)+ \sigma _w^2\varvec{\Psi }(n)E\{\mathbf {\widetilde{x}}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - E\{\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - \varvec{\Psi }(n)E\{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\}\nonumber \\&\quad + \varvec{\Psi }(n)\mathbf {C}(n)\varvec{\Psi }^\mathrm{{T}}(n). \end{aligned}$$

(35)

Here, we show that $\mathbf {C}(n)=\sigma _{\tilde{x}}^{2} \sigma ^2_x \mathrm {tr}\big \{\mathbf {Q}(n)\big \}\mathbf {I}$. The matrix $\mathbf {C}(n)$ can be rewritten as

$$\begin{aligned} \mathbf {C}(n)&= E\big \{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\big \}\nonumber \\&= E\big \{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)E\{\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\}\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\big \}\nonumber \\&= E\big \{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\mathbf {Q}(n)\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\big \}. \end{aligned}$$

(36)

Therefore, if $c_{pq}(n)$ is the element of $(p+1)$th row and $(q+1)$th column of $\mathbf {C}(n)$ and $r_{ij}(n)$ is the element of $(i+1)$th row and $(j+1)$th column of $\mathbf {Q}(n)$, we have

$$\begin{aligned} c_{pq}(n)&= \sum _{i=0}^{L-1}\sum _{j=0}^{L-1}E\left\{ \widetilde{x}_p(n)\widetilde{x}_q(n)x_i(n)x_j(n)\right\} r_{ij}(n)\nonumber \\&\approx E\left\{ \widetilde{x}_p(n)\widetilde{x}_q(n)\right\} \sum _{i=0}^{L-1}\sum _{j=0}^{L-1}E\left\{ x_i(n)x_j(n)\right\} r_{ij}(n). \end{aligned}$$

(37)

The approximation in the above equation is written with assumption that the $i$th element of $\mathbf {\widetilde{x}}(n)$ is independent of $j$th element of $\mathbf {x}(n)$, for $0\le i,j \le L-1$. This assumption is reasonable, since the input signal was assumed to be i.i.d. Gaussian distributed. With a simple calculation, we can see that $\sum _{i=0}^{L-1}\sum _{j=0}^{L-1}E\left\{ x_i(n)x_j(n)\right\} r_{ij}(n)$ is equal to the matrix product of the expression $E\{\mathbf {x}^\mathrm{{T}}(n)\mathbf {Q}(n)\mathbf {x}(n)\}$ and on the other hand, $E\left\{ \widetilde{x}_p(n)\widetilde{x}_q(n)\right\}$ is the $(p+1,q+1)$th element of $\widetilde{\mathbf {R}}(n)$; thus we approximately have

$$\begin{aligned} \mathbf {C}(n)=\widetilde{\mathbf {R}}(n) E\{\mathbf {x}^\mathrm{{T}}(n)\mathbf {Q}(n)\mathbf {x}(n)\}. \end{aligned}$$

(38)

In addition, the scalar expression $E\{\mathbf {x}^\mathrm{{T}}(n)\mathbf {Q}(n)\mathbf {x}(n)\}$ equals $\mathrm {tr}\big \{\mathbf {R}_x(n)\mathbf {Q}(n)\big \}$. Hence,

$$\begin{aligned} \mathbf {C}(n)=\widetilde{\mathbf {R}}(n) \mathrm {tr}\big \{\mathbf {R}_x(n)\mathbf {Q}(n)\big \}. \end{aligned}$$

(39)

As the input signal is assumed i.i.d Gaussian, therefore, $\mathbf {R}_x(n)=\sigma ^2_x\mathbf {I}$. In this case, the $i$th element of $\mathbf {x}(n)$ is independent from $j$th element of $\mathbf {x}(n)$ for $i\ne j$. Therefore, $i$th element of $\mathbf {\widehat{x}}(n)$ is also independent from $j$th element of $\mathbf {\widehat{x}}(n)$ for $i\ne j$. As a result, $\mathbf {\widetilde{R}}(n)=\sigma _{\tilde{x}}^{2}\mathbf {I}$ in which $\sigma _{\tilde{x}}^{2}$ is the variance of clipped input signals. As a result, (39) is simplified to

$$\begin{aligned} \mathbf {C}(n)=\sigma _{\tilde{x}}^{2} \sigma ^2_x \mathrm {tr}\big \{\mathbf {Q}(n)\big \}\mathbf {I}. \end{aligned}$$

(40)

Employing (40), (35) can be written as

$$\begin{aligned} \mathbf {Q}(n+1)&= \mathbf {Q}(n) + \sigma _w^2\varvec{\Psi }(n)\widetilde{\mathbf {R}}(n)\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - E\{\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\}E\{\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - \varvec{\Psi }(n)E\{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\}E\{\mathbf {v}(n)\mathbf {v}^\mathrm{{T}}(n)\}\nonumber \\&\quad + \varvec{\Psi }(n)\sigma _{\tilde{x}}^{2} \sigma ^2_x \mathrm {tr}\big \{\mathbf {Q}(n)\big \}\varvec{\Psi }^\mathrm{{T}}(n) \end{aligned}$$

(41)

in which, similar to [22], we have assumed that $\mathbf {x}(n)$ and $\mathbf {v}(n)$ are independent from each other.

Under steady-state condition, we assume that the adaptive algorithm has converged so that $\varphi (n)$ varies to within an expected value. Therefore,

$$\begin{aligned} \lim _{n \rightarrow \infty } \mathbf {Q}(n+1) = \lim _{n \rightarrow \infty } \mathbf {Q}(n)=\mathbf {Q}. \end{aligned}$$

(42)

Equation (41) can then be rewritten for the steady-state case as

$$\begin{aligned} \mathbf {Q}&= \mathbf {Q} + \sigma _w^2\varvec{\Psi }(n)\widetilde{\mathbf {R}}(n)\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - \mathbf {Q}E\{\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - \varvec{\Psi }(n)E\{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\}\mathbf {Q}\nonumber \\&\quad + \varvec{\Psi }(n)\sigma _{\tilde{x}}^{2} \sigma ^2_x \mathrm {tr}\big \{\mathbf {Q}\big \}\varvec{\Psi }^\mathrm{{T}}(n) \end{aligned}$$

(43)

resulting in

$$\begin{aligned}&\sigma _w^2\varvec{\Psi }(n)\widetilde{\mathbf {R}}(n)\varvec{\Psi }^\mathrm{{T}}(n) - \mathbf {Q}E\{\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad - \varvec{\Psi }(n)E\{\mathbf {\widetilde{x}}(n)\mathbf {x}^\mathrm{{T}}(n)\}\mathbf {Q}+ \varvec{\Psi }(n)\sigma _{\tilde{x}}^{2} \sigma ^2_x \mathrm {tr}\big \{\mathbf {Q}\big \}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&=\mathbf {0}. \end{aligned}$$

(44)

It is shown in [19] that for Gaussian tap-input signal vectors,

$$\begin{aligned} E\big \{\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\big \}=\frac{\alpha }{\sigma _x} E\big \{\mathbf {x}(n)\mathbf {x}^\mathrm{{T}}(n)\big \}, \end{aligned}$$

(45)

where $\alpha =\sqrt{2/\pi }\exp (-\delta ^2/2)$. As a result,

$$\begin{aligned} E\big \{\mathbf {x}(n)\mathbf {\widetilde{x}}^\mathrm{{T}}(n)\big \}={\alpha }{\sigma _x} \mathbf {I}. \end{aligned}$$

(46)

Substituting (45) and (46) into (44), we obtain

$$\begin{aligned}&\sigma _w^2\sigma _{\tilde{x}}^{2}\varvec{\Psi }(n)\varvec{\Psi }^\mathrm{{T}}(n) - {\alpha }{\sigma _x}\mathbf {Q}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad -\, \alpha \sigma _x\varvec{\Psi }(n)\mathbf {Q} + \sigma _{\tilde{x}}^{2} \sigma _x^2 \mathrm {tr}\big \{\mathbf {Q}\big \}\varvec{\Psi }(n)\varvec{\Psi }^\mathrm{{T}}(n)=\mathbf {0}. \end{aligned}$$

(47)

As in C-LMS, $\varvec{\Psi }(n)=\mu \mathbf {I}$, and also in C-RLS, assuming i.i.d. input and $n$ approaches infinity, $\varvec{\Psi }(n)=(1-\lambda )\widetilde{\mathbf {R}}^{-1}(n)=(1-\lambda )\sigma _{\tilde{x}}^{-2}\mathbf {I}$, we derive $\mathbf {Q}\varvec{\Psi }^\mathrm{{T}}(n)= \varvec{\Psi }(n)\mathbf {Q}$. As a result, we can rewrite (47) as

$$\begin{aligned}&\sigma _w^2\sigma _{\tilde{x}}^{2}\varvec{\Psi }(n)\varvec{\Psi }^\mathrm{{T}}(n)- 2{\alpha }{\sigma _x}\mathbf {Q}\varvec{\Psi }^\mathrm{{T}}(n)\nonumber \\&\quad +\, \sigma _{\tilde{x}}^{2} \sigma _x^2 \mathrm {tr}\big \{\mathbf {Q}\big \}\varvec{\Psi }(n)\varvec{\Psi }^\mathrm{{T}}(n)=\mathbf {0} \end{aligned}$$

(48)

which simplifies to

$$\begin{aligned} \sigma _w^2\sigma _{\tilde{x}}^{2}\varvec{\Psi }(n) - 2{\alpha }{\sigma _x}\mathbf {Q} + \sigma _{\tilde{x}}^{2} \sigma _x^2 \mathrm {tr}\big \{\mathbf {Q}\big \}\varvec{\Psi }(n)=\mathbf {0}. \end{aligned}$$

(49)

To evaluate the steady-state misalignment, from (10) and (42), we achieve $\eta =\mathrm {tr}\{\mathbf {Q}\}$. Thus taking the matrix trace of (49) results in

$$\begin{aligned} \sigma _w^2\sigma _{\tilde{x}}^{2}\mathrm {tr}\{\varvec{\Psi }(n)\}-2\alpha \sigma _x\eta +\sigma _{\tilde{x}}^{2} \sigma _x^2\eta \mathrm {tr}\{\varvec{\Psi }(n)\}=0, \end{aligned}$$

(50)

from which we arrive at

$$\begin{aligned} \eta =\frac{\sigma _w^2\sigma _{\tilde{x}}^{2}\mathrm {tr}\{\varvec{\Psi }(n)\}}{2\alpha \sigma _x-\sigma ^2_x\sigma _{\tilde{x}}^{2}\mathrm {tr}\{\varvec{\Psi }(n)\}}. \end{aligned}$$

(51)

Appendix 2

Proof of (12)

According to the definition of the three-level clipping, we have

$$\begin{aligned} {\widetilde{x}^2(n)} =\left\{ \begin{array}{ll} 1, &{} \quad |x(n)|>\delta _x \\ 0, &{} \quad |x(n)|\le \delta _x \end{array} \right. .~ \end{aligned}$$

(52)

The probability that the sample $x(n)$ at instant $n$ put within interval $[-\delta _x , \delta _x]$ is

$$\begin{aligned} P(-\delta _x<x(n)<\delta _x)=\int _{-\delta _x}^{\delta _x}f_x(\tau )\mathrm{{d}}\tau \end{aligned}$$

(53)

where $f_x(\tau )$ is the pdf of $x(n)$. Therefore,

$$\begin{aligned} P(|x(n)|>\delta _x)=1-\int _{-\delta _x}^{\delta _x}f_x(\tau )\mathrm{{d}}\tau =2\int _{\delta _x}^{\infty }f_x(\tau )\mathrm{{d}}\tau , \end{aligned}$$

(54)

which is equal to $P\left( \widetilde{x}(n)=\pm 1\right)$ where in turn equals $P\left( \widetilde{x}^2(n)=1\right)$. i.e.,

$$\begin{aligned} P\left( \widetilde{x}^2(n)=1\right) =P(|x(n)|>\delta _x)=2\int _{\delta _x}^{\infty }f_x(\tau )\mathrm{{d}}\tau . \end{aligned}$$

(55)

On the other hand, with regard to the mathematical definition of the expectation, $E[\cdot ]$, we have

$$\begin{aligned} \sigma _{\tilde{x}}^{2}=E[\widetilde{x}^2(n)]&= 1\times P\left( \widetilde{x}^2(n)=1\right) +0\times P\left( \widetilde{x}^2(n)=0\right) \nonumber \\&= 2\int _{\delta _x}^{\infty }f_x(\tau )\mathrm{{d}}\tau . \end{aligned}$$

(56)

Regarding the assumption of Gaussian input signal,

$$\begin{aligned} \sigma _{\tilde{x}}^{2}=1-\mathrm {erf}\left\{ \frac{\delta _x}{\sigma _x \sqrt{2}}\right\} =\mathrm {erfc}\left\{ \frac{\delta _x}{\sigma _x \sqrt{2}}\right\} =\mathrm {erfc}\left\{ \frac{\delta }{\sqrt{2}}\right\} . \end{aligned}$$

(57)

Thus (12) is proved.