A Switching-Based Variable Step-Size PNLMS Adaptive Filter for Sparse System Identification

Abstract

The standard proportionate normalized least mean square (PNLMS) adaptive algorithm suffers from convergence performance limitation due to a constant step-size during the convergence period. In this paper, a switching-based variable step-size PNLMS is proposed to improve the convergence performance in sparse system identification. To adjust the step-size, the convergence performance of PNLMS is first analysed in the statistical sense and by exploiting the analysis, a switching-based method is then proposed, which brings about a fast convergence towards the desired steady-state mean-square weight deviation. The step-size reduces during the convergence period in a few steps, while in the case of abrupt change in the system impulse response, the step-size increases to its initial value. A sub-band version of the proposed adaptive algorithm is further proposed for highly correlated input signals. Simulation results confirm the superiority of the proposed full-band and sub-band algorithms in terms of convergence performance compared to some competing adaptive algorithms.

Appendices

Appendix A: Stochastic Analysis of PNLMS Convergence Behaviour for Gaussian Inputs

1.1 Transient Analysis

Substituting (17) and (1) into (2) gives

$$\begin{aligned} e(n)&={\textbf{x}}^T(n){\textbf{h}}(n)-{\textbf{x}}^T(n){\textbf{w}}(n)+u(n)\nonumber \\&= -\textbf{x}^T(n)\textbf{v}(n)+u(n) \end{aligned}$$

(20)

where ${\textbf{v}}(n)={\textbf{w}}(n)-{\textbf{h}}(n)$. Subtracting ${\textbf{h}}(n)$ from both sides of (6) and using (20) yields

$$\begin{aligned} \textbf{v}(n+1)=\left\{ {\textbf {I}}-\mu \frac{\textbf{x}(n)\textbf{x}^T(n){\textbf {G}}(n)}{\textbf{x}^T(n){\textbf {G}}(n)\textbf{x}(n)}\right\} \textbf{v}(n)+\mu \frac{u(n){\textbf {G}}(n)\textbf{x}(n)}{\textbf{x}^T(n){\textbf {G}}(n)\textbf{x}(n)}-\textbf{q}(n). \end{aligned}$$

(21)

Then, post-multiplying (21) by its transpose and taking its average yields

$$\begin{aligned} {{\textbf {K}}}_{vv}(n+1)&={{\textbf {K}}}_{vv}(n)-\mu E\left[ \frac{{\textbf{x}}(n){\textbf{x}}^T(n){{\textbf {G}}}(n)}{{\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)}\right] {{\textbf {K}}}_{vv}(n)\nonumber \\&\quad -\mu {{\textbf {K}}}_{vv}(n)E\left[ \frac{{{\textbf {G}}}(n){\textbf{x}}(n){\textbf{x}}^T(n)}{{\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)}\right] \nonumber \\&\quad +\mu ^2E\left[ \frac{{\textbf{x}}(n){\textbf{x}}^T(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n){\textbf{x}}(n){\textbf{x}}^T(n)}{\left( {\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)\right) ^2}\right] \nonumber \\&\quad +\mu ^2E\left[ u^2\frac{{{\textbf {G}}}(n){\textbf{x}}(n){\textbf{x}}^T(n){{\textbf {G}}}(n)}{\left( {\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)\right) ^2}\right] +E\left[ {\textbf{q}}(n){\textbf{q}}^T(n)\right] \end{aligned}$$

(22)

where ${{\textbf {K}}}_{vv}(n)=E\left[ {\textbf{v}}(n){\textbf{v}}^T(n)\right]$. Equation (22) is obtained using independence theory which assumes the weights ${\textbf{w}}(n)$ are statistically independent from ${\textbf{x}}(n)$.

To simplify (22), we define $\tilde{{{\textbf {K}}}}_{vv}(n)={{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)$ and hence the fourth term in the right side of (22) can be simplified as

$$\begin{aligned}&E\left[ \frac{{\textbf{x}}(n){\textbf{x}}^T(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n){\textbf{x}}(n){\textbf{x}}^T(n)}{\left( {\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)\right) ^2}\right] \nonumber \\&\quad =E\left[ \frac{{\textbf{x}}(n){\textbf{x}}^T(n)}{{\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)}\tilde{{{\textbf {K}}}}_{vv}(n)\frac{{\textbf{x}}(n){\textbf{x}}^T(n)}{{\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)}\right] \nonumber \\&\quad =2\tilde{{{\textbf {R}}}}_x(n)\tilde{{{\textbf {K}}}}_{vv}(n)\tilde{{{\textbf {R}}}}_x(n) +\textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n)\tilde{{{\textbf {K}}}}_{vv}(n)\right] \tilde{{{\textbf {R}}}}_x(n) \end{aligned}$$

(23)

where $\tilde{{{\textbf {R}}}}_x(n)=E\left[ \frac{{\textbf{x}}(n){\textbf{x}}^T(n)}{{\textbf{x}}^T(n){{\textbf {G}}}(n){\textbf{x}}(n)} \right]$. The equality in (23) is true for zero-mean Gaussian inputs ${\textbf{x}}(n)$ [35] (Eq. (10.4.26)). As a result, (22) can be rewritten as

$$\begin{aligned} {{\textbf {K}}}_{vv}(n+1)&={{\textbf {K}}}_{vv}(n)-\mu \tilde{{{\textbf {R}}}}_x(n) {{\textbf {G}}}(n) {{\textbf {K}}}_{vv}(n)\nonumber \\&\quad -\mu {{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\tilde{{{\textbf {R}}}}_x(n)\nonumber \\&\quad +2\mu ^2 \tilde{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\tilde{{{\textbf {R}}}}_x(n)\nonumber \\&\quad +\textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\right] \tilde{{{\textbf {R}}}}_x(n)\nonumber \\&\quad +\mu ^2\sigma _u^2 {{\textbf {G}}}(n)\hat{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n)+\sigma _q^2{{\textbf {I}}}. \end{aligned}$$

(24)

To further proceed, we define the MSD criterion as

$$\begin{aligned} \xi (n)=E\left[ ({\textbf{w}}(n)-{\textbf{h}}(n))^T({\textbf{w}}(n)-{\textbf{h}}(n))\right] =\textrm{Tr}\left[ {{\textbf {K}}}_{vv}(n)\right] . \end{aligned}$$

(25)

Thus, taking the trace of both sides of (24) and using the definition in (25) yields

$$\begin{aligned} \xi (n+1)&=\xi (n)-2\mu \textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n) {{\textbf {G}}}(n) {{\textbf {K}}}_{vv}(n)\right] \nonumber \\&\quad +2\mu ^2\textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\tilde{{{\textbf {R}}}}_x(n)\right] \nonumber \\&\quad +\mu ^2\textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\right] \textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n)\right] \nonumber \\&\quad +\mu ^2\sigma _u^2 \textrm{Tr}\left[ {{\textbf {G}}}(n)\hat{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n)\right] +\sigma _q^2 N. \end{aligned}$$

(26)

To proceed with the study of the MSD behaviour of PNLMS, we employ the following approximations

$$\begin{aligned} \textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n) {{\textbf {G}}}(n) {{\textbf {K}}}_{vv}(n)\right]&= \sum _{i=1}^N E\left[ v_i^2(n)\right] g_i(n) {\tilde{\sigma }}_x^2(n-i+1)\nonumber \\&\approx \frac{1}{N}{\tilde{\sigma }}_x^2(n) \textrm{Tr}\left[ {{\textbf {G}}}(n)\right] \xi (n), \end{aligned}$$

(27)

$$\begin{aligned} \textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\tilde{{{\textbf {R}}}}_x(n)\right]&=\sum _{i=1}^NE\left[ v_i^2(n)\right] g_i^2(n){\tilde{\sigma }}_x^4(n-i+1)\nonumber \\&\approx \frac{1}{N}{\tilde{\sigma }}_x^4(n)\textrm{Tr}\left[ {{\textbf {G}}}^2(n)\right] \xi (n), \end{aligned}$$

(28)

$$\begin{aligned} \textrm{Tr}\left[ \tilde{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n){{\textbf {K}}}_{vv}(n){{\textbf {G}}}(n)\right]&= \sum _{i=1}^N E\left[ v_i^2(n)\right] g_i^2(n) {\tilde{\sigma }}_x^2(n-i+1)\nonumber \\&\approx \frac{1}{N}{\tilde{\sigma }}_x^2(n) \textrm{Tr}\left[ {{\textbf {G}}}^2(n)\right] \xi (n), \end{aligned}$$

(29)

$$\begin{aligned} \textrm{Tr}\left[ {{\textbf {G}}}(n)\hat{{{\textbf {R}}}}_x(n){{\textbf {G}}}(n)\right]&= \sum _{i=1}^N g_i^2(n) {\hat{\sigma }}_x^2(n-i+1)\nonumber \\&\approx {\hat{\sigma }}_x^2(n) \textrm{Tr}\left[ {{\textbf {G}}}^2(n)\right] . \end{aligned}$$

(30)

Substituting (27)–(30) into (26) gives

$$\begin{aligned} \xi (n+1)&=s_1(n) \xi (n)+ s_2(n) \end{aligned}$$

(31)

where

$$\begin{aligned} s_1(n)&=1-2\frac{\mu }{N}{\tilde{\sigma }}_x^2(n)\textrm{Tr}\left[ {{\textbf {G}}}(n)\right] +\frac{\mu ^2}{N}(N+2){\tilde{\sigma }}_x^4(n)\textrm{Tr}\left[ {{\textbf {G}}}^2(n)\right] ,\end{aligned}$$

(32)

$$\begin{aligned} s_2(n)&=\mu ^2\sigma _u^2 {\hat{\sigma }}_x^2(n)\textrm{Tr}\left[ {{\textbf {G}}}^2(n)\right] +N\sigma _q^2. \end{aligned}$$

(33)

Equation (31) provides a recursion for MSD of the PNLMS algorithm for both time-invariant and time-varying systems.

1.2 Steady-State Analysis

Proceeding with (31), the steady-state MSD is given by

$$\begin{aligned} \xi _{\textrm{ss}}=\lim _{n\xrightarrow {}\infty }\xi (n)=\frac{s_2}{1-s_1} \end{aligned}$$

(34)

where

$$\begin{aligned} s_1&=\lim _{n\xrightarrow {}\infty }s_1(n)=1-2\frac{\mu }{N}{\tilde{\sigma }}_x^2\textrm{Tr}\left[ {{\textbf {G}}}_\text {ss}\right] +\frac{\mu ^2}{N}(N+2){\tilde{\sigma }}_x^4\textrm{Tr}\left[ {{\textbf {G}}}_\text {ss}^2\right] ,\\ s_2&=\lim _{n\xrightarrow {}\infty }s_2(n)=\mu ^2\sigma _u^2{\hat{\sigma }}_x^2\textrm{Tr}\left[ {{\textbf {G}}}_\text {ss}^2\right] +N\sigma _q^2 \end{aligned}$$

and the matrix ${{\textbf {G}}}_\text {ss}$ denotes the steady-state value of ${{\textbf {G}}}(n)$. In addition, ${\tilde{\sigma }}_x^2$ and ${\hat{\sigma }}_x^2$ are the steady-state values of ${\tilde{\sigma }}_x^2(n)$ and ${\hat{\sigma }}_x^2(n)$.

Appendix B: Derivation of the Step-Size of VSS-PNLMS

For the estimation of the steady-state MSD, $\xi _{\textrm{ss}}$, of PNLMS in (34), ${{\textbf {G}}}_\text {ss}$ is required. Taking into account the fact that ${{\textbf {G}}}_\text {ss}$ is unknown during the convergence of the PNLMS algorithm, the updates of ${{\textbf {G}}}(n)$ at $n_i$ could be used. Thus we define

$$\begin{aligned} \xi _{\textrm{ss},i+1}= \frac{s_{2}(n_{i})}{1-s_{1}(n_{i})},~~ {i=1,\ldots , K-1}. \end{aligned}$$

(35)

Assuming $\sigma _q^2=0$, and substituting (32) and (33) into (35) for $i=1,\ldots , K-1$ and $n_{i-1}< n\le n_i$, yields

$$\begin{aligned} \xi _{\textrm{ss},i+1}&=\frac{N\mu (n)\sigma _u^2{\hat{\sigma }}_x^2(n_{i})/{\tilde{\sigma }}_x^2(n_{i})}{2\textrm{Tr}\left[ {{\textbf {G}}}(n_{i})\right] / \textrm{Tr}\left[ {{\textbf {G}}}^2(n_{i})\right] -(N+2)\mu (n){\tilde{\sigma }}_x^2(n_{i})}, \end{aligned}$$

(36)

subject to the fact that the denominator term in (36) should be positive. As a consequence, the following upper bound for step-size is derived:

$$\begin{aligned} \mu (n) < \frac{2\textrm{Tr}\left[ {{\textbf {G}}}(n_{i}) \right] }{(N+2){\tilde{\sigma }}_x^2(n_{i}) \textrm{Tr}\left[ {{\textbf {G}}}^2(n_{i}) \right] }. \end{aligned}$$

In addition, similar to the derivation of (35) and using (36), $\xi _{\textrm{ss},1}$ can be defined as

$$\begin{aligned} \xi _{\textrm{ss},1}= \frac{s_{2}(0)}{1-s_{1}(0)}=\frac{N\mu _1\sigma _u^2{\hat{\sigma }}_x^2(0)/{\tilde{\sigma }}_x^2(0)}{2\textrm{Tr}\left[ {{\textbf {G}}}(0)\right] / \textrm{Tr}\left[ {{\textbf {G}}}^2(0)\right] -(N+2)\mu _1{\tilde{\sigma }}_x^2(0)}. \end{aligned}$$

(37)

Thus, assuming ${{\textbf {G}}}(0)= {{\textbf {I}}}/N$, (37) is simplified to

$$\begin{aligned} \xi _{\textrm{ss},1}&=\frac{N\mu _1\sigma _u^2{\hat{\sigma }}_x^2(0)/{\tilde{\sigma }}_x^2(0)}{2N-(N+2)\mu _1{\tilde{\sigma }}_x^2(0)}. \end{aligned}$$

Therefore, knowing $\xi _{\textrm{ss}}$ and $\xi _{\textrm{ss,1}}$, the value of $\xi _{\textrm{ss},i}$ and accordingly, $n_i$ for all transients will be calculated using (7) and (8), respectively.

Solving (36) for $\mu (n)$, yields

$$\begin{aligned} \mu (n)=\frac{2\xi _{\textrm{ss},{i+1}} \textrm{Tr}\left[ {{\textbf {G}}}(n_i)\right] / \textrm{Tr}\left[ {{\textbf {G}}}^2(n_i)\right] }{N\sigma _u^2{\hat{\sigma }}_x^2(n_i)/{\tilde{\sigma }}_x^2(n_i)+(N+2) {\tilde{\sigma }}_x^2(n_i)\xi _{\textrm{ss},{i+1}}}. \end{aligned}$$

Besides, in time-varying scenarios when $\sigma _q^2\ne 0$, in a similar manner, substituting (32) and (33) into (35) and solving (35) for $\mu (n)$ in the interval $n_{i-1}<n\le n_i$ and $i=2,\ldots ,K$, yields

$$\begin{aligned} \mu (n)&=\frac{ \xi _{\textrm{ss},{i+1}} \textrm{Tr}\left[ {{\textbf {G}}}(n_i)\right] + \sqrt{ \xi _{\textrm{ss},i}^2 \textrm{Tr}\left[ {{\textbf {G}}}(n_i)\right] ^2 - N^2\frac{\sigma _q^2}{{\tilde{\sigma }}_x^2(n_i)}\textrm{Tr}\left[ {{\textbf {G}}}^2(n_i)\right] c(n_i)} }{\textrm{Tr}\left[ {{\textbf {G}}}^2(n_i)\right] c(n_i)} \end{aligned}$$

where

$$\begin{aligned} c(n_i)= N \frac{\sigma _u^2{\hat{\sigma }}_x^2(n_i)}{ {\tilde{\sigma }}_x^2(n_i)} +(N+2){\tilde{\sigma }}_x^2(n_i) \xi _{\textrm{ss},{i+1}}. \end{aligned}$$