Analysis of Convergence of a Frequency-Domain LMS Adaptive Filter Implemented as a Multi-Stage Adaptive Filter

Abstract

We present an analysis of the convergence of the frequency-domain LMS adaptive filter when the DFT is computed using the LMS steepest descent algorithm. In this case, the frequency-domain adaptive filter is implemented with a cascade of two sections, each updated using the LMS algorithm. The structure requires less computations compared to using the FFT and is modular suitable for VLSI implementations. Since the structure contains two adaptive algorithms updating in parallel, an analysis of the overall system convergence needs to consider the effect of the two adaptive algorithms on each other, in addition to their individual convergence. Analysis was based on the expected mean-square coefficient error for each of the two LMS adaptive algorithms, with some simplifying approximations for the second algorithm, to describe the convergence behavior of the overall system. Simulations were used to verify the results.

Appendix

1.1 Detailed convergence analysis

These are the Error and Coefficient update Equations

For (Time-Domain) LMS-DFT

Input: X (k)

Error: e ₁(k) = c(k) − H ^H(k)X(k)

Update:

$$H\left( {k + 1} \right) = H\left( k \right) + 2\mu _H X\left( k \right)e_1^* \left( k \right)$$

For (Frequency Domain) LMS

Input: Z(k) = H(k)⊗X(k)

Error: e ₂(k) = d(k) − W ^H(k)Z(k)

Update: W(k + 1) = W(k) + 2μ _w Z(k)e ^*(k)

For (Time-Domain) LMS-DFT(1^st LMS Algorithm)

Error: $e_1 \left( k \right) = \underbrace {\left( {H_0^H X\left( k \right) + n_1 \left( k \right)} \right)}_{c\left( k \right)} - H^H \left( k \right)X\left( k \right)$ where H ₀ (optimal solution) is

$$\matrix {{\text{stationary}}} { = \left( {H_0^H - H^H } \right)X\left( k \right) + n_1 \left( k \right)} {} \\ {} { = - ^ \wedge H^H X\left( k \right) + n_1 \left( k \right)} {{\text{where }}^ \wedge H = \left( {H - H_0 } \right)\quad {\text{is}}\;{\text{a}}\;{\text{weight}}\;{\text{error}}\;{\text{vector}}} \ \matrix {} \\ {\quad \quad } \ $$

$$\matrix {{\text{Update:}}} \hfill {H\left( {k + 1} \right) = H\left( k \right) + 2\mu _H X\left( k \right)e_1^* \left( k \right)} \hfill \\ {} \hfill {\left( {H\left( {k + 1} \right) - H_0 } \right) = \left( {H\left( k \right) - H_0 } \right) + 2\mu _H X\left( k \right)\underbrace {\left( { - ^ \wedge H^H \left( k \right)X\left( k \right) + n_1 \left( k \right)} \right)}_{e_1 \left( k \right)}^* } \hfill \\ {} \hfill {^ \wedge H\left( {k + 1} \right) = ^ \wedge H\left( k \right) + 2\mu _H X\left( k \right)\left( { - ^ \wedge H^T \left( k \right)X^* \left( k \right) + n_1^* \left( k \right)} \right)} \hfill \\ {} \hfill {^ \wedge H\left( {k + 1} \right) = ^ \wedge H\left( k \right) + 2\mu _H X\left( k \right)\left( { - X^H \left( k \right)^ \wedge H\left( k \right) + n_1^* \left( k \right)} \right)} \hfill \\ {} \hfill {^ \wedge H\left( {k + 1} \right) = ^ \wedge H\left( k \right) - 2\mu _H X\left( k \right)X^H \left( k \right)^ \wedge H\left( k \right) + 2\mu _H X\left( k \right)n_1^* \left( k \right)} \hfill \\ {} \hfill {^ \wedge H\left( {k + 1} \right) = \left( {I - 2\mu _H X\left( k \right)X^H \left( k \right)} \right)^ \wedge H\left( k \right) + 2\mu _H X\left( k \right)n_1^* \left( k \right)} \hfill \ $$

Let ${}^ \wedge{H}^1 \left( k \right) = Q^H {}^ \wedge{H}\left( k \right)$, where ${}^ \wedge{H}H^1 \left( k \right)$ is in rotated co-ordinates

$$\matrix {{\text{Then}}} \hfill {Q^H \left[ {^ \wedge H\left( {k + 1} \right)} \right] = Q^H \left[ {\left( {I - 2\mu _H X\left( k \right)X^H \left( k \right)} \right)^ \wedge H\left( k \right) + 2\mu _H X\left( k \right)n_1^* \left( k \right)} \right]} \hfill \\ {} \hfill {^ \wedge H^1 \left( {k + 1} \right) = \left[ {\left( {Q^H - 2\mu _H Q^H X\left( k \right)X^H \left( k \right)} \right)QQ^{H} {^ \wedge {H}}\left( k \right) + 2\mu _H Q^H X\left( k \right)n_1^* \left( k \right)} \right]} \hfill \\ {} \hfill {^ \wedge H^1 \left( {k + 1} \right) = \left[ {\left( {I - 2\mu _H X^1 \left( k \right)X^{1H} \left( k \right)} \right)^ \wedge H^1 \left( k \right) + 2\mu _H X^1 \left( k \right)n_1^* \left( k \right)} \right]} \hfill \ $$

For (Frequency Domain) LMS (2 ^nd LMS Algorithm)

Error:$\matrix {e_{2} \left( k \right)}{ = \underbrace {\left( {W_{0}^H Z\left( k \right) + n_{2} \left( k \right)} \right)}_{d\left( k \right)} - W^H \left( k \right)Z\left( k \right)} \\ {}{ = - {}^ \wedge{W}^H Z\left( k \right) + n_{2} \left( k \right)} \$where W ₀ (optimal solution) is stationary

$$\matrix {{\text{Update:}}} \hfill {W(k + 1)} \hfill { = W(k) + 2\mu _w Z(k)e_2^* (k)} \hfill \\ {} \hfill {^ \wedge W^1 (k + 1)} \hfill { = \left[ {\left( {I - 2\mu _w Z^1 (k)Z^{1H} (k)} \right)^ \wedge W^1 (k) + 2\mu _w Z^1 (k)n_2^* (k)} \right]} \hfill \ $$

where Z(k) = H(k) ⊗ X(k)

Recall that ${}^nV\prime _{11} \left( n \right) = E_{nx} \left[ {{}^nH\prime ^H \left( n \right) \otimes {}^nH\prime \left( n \right)} \right]$ Also recall that ${}^nC_{11}^\prime \left( k \right)$, ${}^nC_{22}^\prime \left( k \right)$ are the equivalent MSE matrices for ${}^ \wedge{V}_{11} \left( k \right)$, ${}^ \wedge{V}_{22} \left( k \right)$. They are just the diagonals of ${}^nC_{11}^\prime \left( k \right)$, ${}^nC_{22}^\prime \left( k \right)$.

$$\begin{array}{*{20}l} {{{\text{To}}\;{\text{find:}}} \hfill} & {{^{ \wedge } V_{{11}} {\left( k \right)} = E{\left[ {^{ \wedge } H\prime ^{H} {\left( k \right)} \otimes ^{ \wedge } H\prime {\left( k \right)}} \right]}} \hfill} \\ {{} \hfill} & {{^{ \wedge } H\prime {\left( {k + 1} \right)} = {\left( {I - 2\mu _{H} X\prime {\left( k \right)}X\prime ^{{\rm H}} {\left( k \right)}} \right)}^{ \wedge } H\prime {\left( k \right)} + 2\mu _{H} X\prime {\left( k \right)}n^{*}_{1} {\left( k \right)}} \hfill} \\ {{} \hfill} & {{{}^{n}C^{\prime }_{{11}} {\left( {k + 1} \right)} = E_{{nx}} {\left[ {^{ \wedge } H^{1} {\left( {k + 1} \right)}^{ \wedge } H^{{1H}} {\left( {k + 1} \right)}} \right]}} \hfill} \\ \end{array}$$

Substitute ${}^ \wedge{H}^1 \left( {k + 1} \right) = \left[ {\left( {I - 2\mu _H X^1 \left( k \right)X^{1H} \left( k \right)} \right){}^ \wedge{H}^1 \left( k \right) + 2\mu _H X^1 \left( k \right)n_1^* \left( k \right)} \right]$ to get

$$\begin{array}{*{20}c} {{}^{n}C^{\prime }_{{11}} {\left( {k + 1} \right)}}{ = E_{{nx}} {\left[ {{\left[ {{\left( {I - 2\mu _{H} X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right)}{}^{ \wedge }H^{1} {\left( k \right)} + 2\mu _{H} X^{1} {\left( k \right)}n^{*}_{1} {\left( k \right)}} \right]}{\left[ {{\left( {I - 2\mu _{H} X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right)}{}^{ \wedge }H^{1} {\left( k \right)} + 2\mu _{H} X^{1} {\left( k \right)}n^{*}_{1} {\left( k \right)}} \right]}} \right]}} \\ {}{ = E_{{nx}} {\left[ {{\left[ {{\left( {I - 2\mu _{H} X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right)}{}^{ \wedge }H^{1} {\left( k \right)} + 2\mu _{H} X^{1} {\left( k \right)}n^{*}_{1} {\left( k \right)}} \right]}{\left[ {{}^{ \wedge }H^{{1H}} {\left( k \right)}{\left( {I - 2\mu _{H} X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right)}^{{1H}} + 2\mu _{H} X^{{1H}} {\left( k \right)}n^{*}_{1} {\left( k \right)}} \right]}} \right]}} \\ {}{ = E_{{nx}} {\left[ {{\left[ {{\left( {I - 2\mu _{H} X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right)}{}^{ \wedge }H^{1} {\left( k \right)}{}^{ \wedge }H^{{1H}} {\left( k \right)}{\left( {I - 2\mu _{H} X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right)}} \right]}} \right]}} \\ {}{{\left[ { + 2\mu _{H} {\left( {I - 2\mu _{H} X\prime {\left( k \right)}X^{{\prime H}} {\left( k \right)}} \right)}{}^{ \wedge }H\prime {\left( k \right)}{}^{ \wedge }X^{{1H}} {\left( k \right)}n_{1} {\left( k \right)} + 2\mu _{H} n^{*}_{1} {\left( k \right)}X^{1} {\left( k \right)}{}^{ \wedge }H^{{1H}} {\left( k \right)}{\left( {I - 2\mu _{H} X\prime {\left( k \right)}X^{{\prime H}} {\left( k \right)}} \right)}} \right]}{\left[ { + 4\mu ^{2}_{H} n^{*}_{1} {\left( k \right)}n_{1} {\left( k \right)}X^{1} {\left( k \right)}X^{{1H}} {\left( k \right)}} \right]}} \\ \end{array}$$

Since n₁(k) is independent of X(k),H(k) and E[n ₁(k)] = ϕ

$$\begin{array}{*{20}c}{{}^nC_{11}^\prime \left( {k + 1} \right)}{ = E_x \left[ {\left( {I - 2\mu _H X^\prime \left( k \right)X^\prime{H} \left( k \right)} \right)^\wedge C_{11}^\prime \left( k \right)\left( {I - 2\mu _H X^\prime \left( k \right)X^\prime{H} \left( k \right)} \right)} \right] + 4\mu _H^2 \underbrace {E\left[ {n_1^* (k)n_1 (k)} \right]}_{\sigma _n^2 }\underbrace {E_x \left[ {X^\prime (k)X^\prime{H} (k)} \right]}_\Delta } \\{}{ = E\left[ {^\wedge C_{11}^\prime \left( k \right)} \right] - 2\mu _H E_x \left[ {^\wedge C_{11}^\prime \left( k \right)X^\prime \left( k \right)X^\prime{H} \left( k \right)} \right] - 2\mu _H E_x \left[ {X^\prime \left( k \right)X^\prime{H} \left( k \right)^\wedge C_H^\prime \left( k \right)} \right] + 4\mu _H^2 E_x \left[ {X^\prime \left( k \right)X^\prime{H} \left( k \right)^\wedge C_{11}^\prime \left( k \right)X^\prime \left( k \right)X^\prime{H} \left( k \right)} \right] + 4\mu _H^2 \sigma _{n1}^2 \Delta } \\{}{ = ^\wedge C_{11}^\prime \left( k \right) - 2\mu _H ^\wedge C_{11}^\prime \left( k \right)\Delta - 2\mu _H \Delta ^\wedge C_{11}^\prime \left( k \right) + 4\mu _H^2 E_x \left[ {X^\prime \left( k \right)X^\prime{H} \left( k \right)^\wedge C_{11}^\prime \left( k \right)X^\prime \left( k \right)X^\prime{H} \left( k \right)} \right] + 4\mu _H^2 \sigma _{n1}^2 \Delta } \\\end{array} $$

$$\matrix {{\text{Since}}} {E_x \left[ {X\prime \left( k \right)X\prime ^H \left( k \right)SX\prime \left( k \right)X\prime ^H \left( k \right)} \right] = \Delta S\Delta + tr\left( {S\Delta } \right)\Delta } \ $$

$$^\wedge C_{11}^\prime \left( {k + 1} \right) = ^\wedge C_{11}^\prime \left( k \right) - 2\mu _H ^\wedge C_{11}^\prime \left( k \right)\Delta - 2\mu _H ^\wedge C_{11}^\prime \left( k \right)\Delta + 4\mu _H^2 \Delta ^\wedge C_{11}^\prime \left( k \right)\Delta + 4\mu _H^2 tr\left( {C_{11}^\prime \left( k \right)\Delta } \right)\Delta + 4\mu _H^2 \sigma _{n1}^2 \Delta $$

So diagonal elements are:

$$^{\hat{}} C^{'}_{{11,ii}} {\left( {k + 1} \right)} = {\left( {1 - 4\mu _{H} \lambda _{i} + 4\mu ^{2}_{H} \lambda ^{2}_{i} } \right)}^{\hat{}} C^{'}_{{11,ii}} {\left( k \right)} + 4\mu ^{2}_{H} \lambda _{i} {\sum\limits_{p = 1}^N {\lambda _{p} ^{\hat{}} C^{'}_{{11,pp}} {\left( k \right)}} } + 4\mu ^{2}_{H} \sigma ^{2}_{{n1}} \lambda _{i}$$

Therefore, we can define

$$^\wedge V_{11}^\prime \left( {k + 1} \right) = \phi ^\wedge V_{11}^\prime \left( k \right) + 4\mu _H^2 \sigma _{n1}^2 \lambda $$

$$\matrix {{\text{where}}} {\phi = I - 4\mu _H \Delta + 4\mu _H^2 \Delta ^2 + 4\mu _H^2 \lambda \lambda ^T } \ $$

Now to find

$$^ \wedge V_{22}^\prime \left( k \right)$$

Also recall that ${}^nV\prime_{22} \left( n \right) = E_{nx} [{}^nW\prime^H \left( n \right) \otimes {}^nW\prime\left( n \right)]$

$$\matrix {{\text{To}}\;{\text{find}}} \hfill {{}^ \wedge V_{22}^\prime \left( k \right)} \hfill { = E_{nx} \left[ {{}^ \wedge W^1 \left( k \right) \otimes {}^ \wedge W^{1H} \left( k \right)} \right]} \hfill \\ {} \hfill {{}^ \wedge W^1 \left( {k + 1} \right)} \hfill { = \left[ {\left( {I - 2\mu _H Z^1 \left( k \right)Z^{1H} \left( k \right)} \right){}^ \wedge W^1 \left( k \right) + 2\mu _H Z^1 \left( k \right)Z^{1H} \left( k \right)n_2^* \left( k \right)} \right]} \hfill \\ {} \hfill {{}^nC_{22}^\prime \left( {k + 1} \right)} \hfill { = E_{nx} \left[ {{}^ \wedge W^1 \left( {k + 1} \right){}^ \wedge W^{1H} \left( {k + 1} \right)} \right]} \hfill \ $$

$$\begin{array}{*{20}c} {{}^nC_{22}^\prime \left( {k + 1} \right)}{ = E_{nx} \left[ {\left[ {\left( {I - 2\mu _{W} Z^1 \left( k \right)Z^{1H} \left( k \right)} \right){}^ \wedge W^1 \left( k \right) + 2\mu _{W} Z^1 \left( k \right)n_{2}^* \left( k \right)} \right]\left[ {\left( {I - 2\mu _{W} Z^1 \left( k \right)Z^{1H} \left( k \right)} \right){}^ \wedge W^1 \left( k \right) + 2\mu _W Z^1 \left( k \right)n_{2}^* \left( k \right)} \right]^H } \right]} \\ {}{ = E_{nx} \left[ {\left[ {\left( {I - 2\mu _{W} Z^1 \left( k \right)Z^{1H} \left( k \right)} \right){}^ \wedge W^1 \left( k \right) + 2\mu _{W} Z^1 \left( k \right)n_{2}^* \left( k \right)} \right]\left[ {{}^ \wedge W^{1H} \left( k \right)\left( {I - 2\mu _{W} Z^1 \left( k \right)Z^{1H} \left( k \right)} \right)^{1H} + 2\mu _{W} Z^{1H} \left( k \right)n_{2}^* \left( k \right)} \right]} \right]} \\ {}{ = E_{nx} \left[ {\left[ {\left( {I - 2\mu _{W} Z^1 \left( k \right)Z^{1H} \left( k \right)} \right){}^ \wedge W^1 \left( k \right){}^ \wedge W^{1H} \left( k \right)\left( {I - 2\mu _{W} Z^1 \left( k \right)Z^{1H} \left( k \right)} \right)} \right]} \right]\left[ { + 2\mu _{W} \left( {I - 2\mu _{W} Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right){}^ \wedge W^\prime \left( k \right){}^ \wedge Z^{1H} \left( k \right)n_{2} \left( k \right) + 2\mu _{W} n_2^* \left( k \right)Z^1 \left( k \right)^\wedge W^{1H} \left( k \right)\left( {I - 2\mu _W Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right)} \right]\left[ { + 4\mu _W^2 n_2^* \left( k \right)n_2 \left( k \right)Z^1 \left( k \right)Z^{1H} \left( k \right)} \right]} \\\end{array} $$

Since n ₂(k) is independent of W(k), Z(k) and $E\left[ {n_2 (k)} \right] = \phi$

$$ = E_x \left[ {\left( {I - 2\mu _W Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right)^ \wedge W\prime \left( k \right)W^{\prime H} \left( k \right)\left( {I - 2\mu _W Z^\prime \left( k \right)Z^{'H} \left( k \right)} \right)} \right] + 4\mu _W^2 \sigma _{n2}^2 E_x \left[ {Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right]$$

$${\text{Let}}\;\;A = E_x \left[ {Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right]$$

$${\text{Let}}\;B = E_x \left[ {\left( {I - 2\mu _W Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right)^ \wedge W\prime \left( k \right)W^{\prime H} \left( k \right)\left( {I - 2\mu _W Z^\prime \left( k \right)Z^{\prime {\rm H}} \left( k \right)} \right)} \right]$$

$$\matrix {A = E_x \left[ {Z^\prime \left( k \right)Z^\prime{H} \left( k \right)} \right]}{ = E_x \left[ {\left[ {Q^H \left( {H(k) \otimes X(k)} \right)} \right]\left[ {Q^H \left( {H(k) \otimes X(k)} \right)} \right]^H } \right]} \\ {}{ = E_x \left[ {\left[ {Q^H \left( {X(k) \otimes H(k)} \right)} \right]\left[ {\left( {H^H (k) \otimes X^H (k)} \right)Q} \right]} \right]} \\ {}{ = E_x \left[ {\left( {X^\prime (k) \otimes H(k)} \right)\left( {H^H (k) \otimes X^\prime{H} (k)} \right)} \right]} \\ {}{ = E_x \left[ {\left( {H(k) \otimes X^\prime (k)} \right)\left( {X^\prime{H} (k) \otimes H^H (k)} \right)} \right]} \ $$

Convert H(k), H ^H(k) to diagonal matrices

$$\begin{aligned} & = E_{x} {\left[ {{\left( {H_{d} {\left( k \right)}X\prime {\left( k \right)}} \right)}{\left( {X\prime ^{H} {\left( k \right)}H^{H}_{d} {\left( k \right)}} \right)}} \right]} \\ & = E_{x} {\left[ {{\left( {H_{d} {\left( k \right)}X\prime {\left( k \right)}X\prime ^{H} {\left( k \right)}H^{H}_{d} {\left( k \right)}} \right)}} \right]} \\ & = E{\left[ {H_{d} {\left( k \right)}} \right]}E_{x} {\left[ {X\prime {\left( k \right)}X\prime ^{H} {\left( k \right)}} \right]}E{\left[ {H^{H}_{d} {\left( k \right)}} \right]} \\ & = E{\left[ {H_{d} {\left( k \right)}} \right]}\Delta E{\left[ {H^{H}_{d} {\left( k \right)}} \right]} \\ & = B_{1} {\left( k \right)}\Delta B_{1} {\left( k \right)}\quad {\text{where}}\quad B_{1} {\left( k \right)} = E{\left[ {H_{d} {\left( k \right)}} \right]} \\ \end{aligned} $$

$$\begin{array}{*{20}c} {{\text{Recall}}\;{\text{that}}}{B = E_x \left[ {\left( {I - 2\mu _W Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right)^ \wedge W\prime \left( k \right)W\prime ^H \left( k \right)\left( {I - 2\mu _W Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right)} \right]} \\ {}{ = E\left[ {{}^ \wedge W\prime \left( k \right){}^ \wedge W\prime ^H \left( k \right)} \right] - 2\mu _W E_x \left[ {{}^ \wedge W\prime \left( k \right){}^ \wedge W\prime ^H \left( k \right)Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right] - 2\mu _W E_x \left[ {Z\prime \left( k \right)Z\prime ^H \left( k \right){}^ \wedge W\prime \left( k \right){}^ \wedge W\prime ^H \left( k \right)} \right] + 4\mu _W^2 E_x \left[ {Z\prime \left( k \right)Z\prime ^H \left( k \right){}^ \wedge W\prime \left( k \right){}^ \wedge W\prime ^H \left( k \right)Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right]} \\ {}{ = {}^ \wedge C_{22}^\prime \left( k \right) - 2\mu _W {}^ \wedge C_{22}^\prime \left( k \right)E\left[ {Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right] - 2\mu _W {}^ \wedge C_{22}^\prime \left( k \right)E\left[ {Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right] + 4\mu _W^2 E\left[ {Z\prime \left( k \right)Z\prime ^H \left( k \right){}^ \wedge W\prime \left( k \right){}^ \wedge W\prime ^H \left( k \right)Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right]} \\ {}{ = {}^ \wedge C_{22}^\prime \left( k \right) - 2\mu _W {}^ \wedge C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta B_1^H \left( k \right) - 2\mu _W {}^ \wedge C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta B_1^H \left( k \right) + 4\mu _W^2 E_x \left[ {Z\prime \left( k \right)Z\prime ^H \left( k \right){}^ \wedge W\prime \left( k \right){}^ \wedge W\prime ^H \left( k \right)Z\prime \left( k \right)Z\prime ^H \left( k \right)} \right]} \\\end{array} $$

$${\text{Let}}\;C = E_x \left[ {Z^\prime \left( k \right)Z^{\prime H} \left( k \right){}^ \wedge W^\prime \left( k \right){}^ \wedge W^{\prime H} \left( k \right)Z^\prime \left( k \right)Z^{\prime H} \left( k \right)} \right]$$

$$\begin{array}{*{20}c} {E_{x} {\left[ {Z^{\prime } {\left( k \right)}Z^{{\prime H}} {\left( k \right)}{}^{ \wedge }W^{\prime } {\left( k \right)}{}^{ \wedge }W^{{\prime H}} {\left( k \right)}Z^{\prime } {\left( k \right)}Z^{{\prime H}} {\left( k \right)}} \right]}}{ = E{\left[ {H_{d} {\left( k \right)}X^{\prime } {\left( k \right)}X^{{\prime H}} {\left( k \right)}H^{H}_{d} {\left( k \right)}{}^{ \wedge }W^{\prime } {\left( k \right)}{}^{ \wedge }W^{{\prime H}} {\left( k \right)}H_{d} {\left( k \right)}X^{\prime } {\left( k \right)}X^{{\prime H}} {\left( k \right)}H^{H}_{d} {\left( k \right)}} \right]}} \\ {}{ = E{\left[ {H_{d} {\left( k \right)}} \right]}E{\left[ {X^{\prime } {\left( k \right)}X^{{\prime H}} {\left( k \right)}H^{H}_{d} {\left( k \right)}{}^{ \wedge }W^{\prime } {\left( k \right)}{}^{ \wedge }W^{{\prime H}} {\left( k \right)}X^{\prime } {\left( k \right)}X^{{\prime H}} {\left( k \right)}H^{H}_{d} {\left( k \right)}} \right]}E{\left[ {H^{H}_{d} {\left( k \right)}} \right]}} \\ \end{array} $$

$$\matrix {{\text{Since}}} \hfill {E_x \left[ {X\prime \left( k \right)X\prime ^H \left( k \right)SX\prime \left( k \right)X\prime ^H \left( k \right)} \right]} \hfill { = \Delta S\Delta + tr\left( {S\Delta } \right)\Delta } \hfill \\ {} \hfill {} \hfill { = B_1 \left( k \right)\Delta B_1^H \left( k \right)^ \wedge C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta B_1^H \left( k \right) + B_1 \left( k \right)tr\left( {B_1^H \left( k \right)^ \wedge C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta } \right)\Delta B_1^H \left( k \right)} \hfill \ $$

Recombining A, B and C

$$\begin{array}{*{20}c}{^\wedge C_{22}^\prime \left( {k + 1} \right)}{ = ^\wedge C_{22}^\prime \left( k \right) - 2\mu _W ^\wedge C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta B_1^H \left( k \right) - 2\mu _W B_1 \left( k \right)\Delta B_1^H \left( k \right)^\wedge C_{22}^\prime \left( k \right)} \\{}{ + 4\mu _W^2 B_1 \left( k \right)\Delta B_1^H \left( k \right)^\wedge C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta B_1^H \left( k \right) + 4\mu _W^2 B_1 \left( k \right)tr\left( {B_1^H \left( k \right)C_{22}^\prime \left( k \right)B_1 \left( k \right)\Delta } \right)\Delta B_1^H \left( k \right)} \\{}{ + 4\mu _W^2 \sigma _{n2}^2 B_1 \left( k \right)\Delta B_1^H \left( k \right)} \\\end{array} $$

So diagonal elements are:

$$^\wedge C_{22,ii}^\prime \left( {k + 1} \right) = \left( {1 - 4\mu _w b_{1,i} b_{1,i}^* \lambda _i + 4\mu _w^2 b_{1,i}^2 b_{1,i}^{*2} \lambda _i^2 } \right) + 4\mu _w^2 b_{1,i} b_{1,i}^* \lambda _i \sum\limits_{p = 1}^N {b_{1,i} b_{1,i}^* \lambda _i ^\wedge C_{22,pp}^\prime \left( k \right)} + 4\mu _w^2 \sigma _{n2}^2 b_{1,i} b_{1,i}^* \lambda _i $$

Therefore, we can define

$$^\wedge V_{22}^\prime \left( {k + 1} \right) = \phi _2 ^\wedge V_{22}^\prime \left( k \right) + 4\mu _w^2 \sigma _{n2}^2 B_1 \left( k \right)B_1^H \left( k \right)\lambda $$

$$\begin{array}{*{20}c} {{\text{where}}} {\phi _2 = I - 4\mu _w B_1 \left( k \right)B_1^H \left( k \right)\Delta + 4\mu _w^2 B_1^2 \left( k \right)B_1^{H2} \left( k \right)\Delta ^2 + 4\mu _w^2 B_1 \left( k \right)B_1^H \left( k \right)\lambda \lambda ^T B_1 \left( k \right)B_1^H \left( k \right)} \\\end{array} $$

Find B ₁(k) for $^\wedge V_{22}^\prime \left( k \right)$

$$\begin{array}{*{20}c} {{{\text{Find}}}} & {{B_{1} {\left( k \right)} = E_{{nx}} {\left[ {H_{d} {\left( k \right)}} \right]}}} \\ \end{array} $$

The weight update algorithm is

$$H\left( {k + 1} \right) = H\left( k \right) + 2\mu _H X\left( k \right)e_1^* \left( k \right)$$

The error is computed as

$$\matrix {e_1^* (k)}{ = \left( {H_0^H X\left( k \right) - H^H \left( k \right)X\left( k \right) + n_1 \left( k \right)} \right)^* } \\ {}{ = \left( {H_0^T X^H \left( k \right) - H^T \left( k \right)X^* \left( k \right) + n_1^* \left( k \right)} \right)} \\ {}{ = \left( {X^H \left( k \right)H_0 - X^H \left( k \right)H\left( k \right) + n_1^* \left( k \right)} \right)} \\ {}{ = X^H \left( k \right)\left( {H_0 - H\left( k \right)} \right) + n_1^* \left( k \right)} \ $$

Therefore

$$\begin{aligned} H\left( {k + 1} \right) = H\left( k \right) + 2\mu _H X\left( k \right)\left[ {X^H \left( k \right)\left( {H_0 - H\left( k \right)} \right) + n_1^* \left( k \right)} \right] \\ H\left( {k + 1} \right) = H\left( k \right) + 2\mu _H X\left( k \right)X^H \left( k \right)\left( {H_0 - H\left( k \right)} \right) + 2\mu _H X\left( k \right)n_1^* \left( k \right) \\ H\left( {k + 1} \right) = H\left( k \right) - 2\mu _H X\left( k \right)X^H \left( k \right)H\left( k \right) + 2\mu _H X\left( k \right)X^H \left( k \right)H_0 + 2\mu _H X\left( k \right)n_1^* \left( k \right) \\ H\left( {k + 1} \right) = \left( {I - 2\mu _H X\left( k \right)X^H \left( k \right)} \right)H\left( k \right) + 2\mu _H X\left( k \right)X^H \left( k \right)H_0 + 2\mu _H X\left( k \right)n_1^* \left( k \right) \\ \end{aligned} $$

$$\matrix {{\text{Let}}} \hfill {b_1 \left( {k + 1} \right)} \hfill { = E_{nx} \left[ {H\left( {k + 1} \right)} \right]} \hfill \\ {} \hfill {} \hfill { = E_x \left[ {\left( {I - 2\mu _H X(k)X^H (k)} \right)H(k)} \right] + 2\mu _H E_x \left[ {X(k)X^H (k)H_0 } \right] + 2\mu _H E_x \left[ {X(k)n_1^* (k)} \right]} \hfill \ $$

Since n ₁(k) is independent of X(k)

$$E_x \left[ {n_1 \left( k \right)} \right] = \phi $$

$$\matrix {b_1 \left( {k + 1} \right)}{ = I - 2\mu _H \left( {E_x \left[ {X\left( k \right)X^H \left( k \right)} \right]} \right)E_x \left[ {H\left( k \right)} \right] + 2\mu _H E_x \left[ {X(k)X^H (k)} \right]H_0 } \\ {}{ = \left( {I - 2\mu _H R} \right)b_1 \left( k \right) + 2\mu _H RH_0 \quad R = \left( {E_x \left[ {X\left( k \right)X^H \left( k \right)} \right]} \right)} \ $$

$$\matrix {{\text{So}}}, {b_1 \left( k \right) = \left( {I - 2\mu _H R} \right)b_1 \left( {k - 1} \right) + 2\mu _H RH_0 } \ $$

And B ₁(k) is diagonal matrix with main diagonal b ₁(k)

Note that B ₁(k) is a diagonal matrix (since it is based on a diagonal version of H). And b ₁(k) is the same as B ₁(k), except in column vector form (since this is the familiar form for finding the expectation of H). Thus, B ₁(k) = diag(b ₁(k)).

$$\begin{array}{*{20}c} {{{\text{Now, to find}}}} & {{E{\left[ {X\prime X\prime ^{H} SX\prime X\prime ^{H} } \right]}}} \\ \end{array}$$

$$\begin{array}{*{20}c}{E_x \left[ {X^\prime \left( k \right)X^\prime{H} \left( k \right)SX^\prime \left( k \right)X^\prime{H} \left( k \right)} \right]}{ = \sum\limits_p {\sum\limits_q {E_x \left[ {X_k^\prime X_p^{\prime*} S_{pq} X_q^\prime X_l^{\prime*} } \right]} } } \\{}{ = \sum\limits_p {\sum\limits_q {\left[ {E_x \left[ {X_k^\prime X_p^{\prime*} } \right]S_{pq} E_x \left[ {X_q^\prime X_l^{\prime*} } \right] + E_x \left[ {X_k^\prime X_l^{\prime*} } \right]S_{pq} E_x \left[ {X_p^{\prime*} X_q^\prime } \right]} \right]} } } \\\end{array} $$

But $E_x \left[ {X_p^{\prime*} X_q^\prime } \right]$ is zero for all p ≠ q, Since $E_x \left[ {X^\prime X^\prime{H} } \right] = \Delta$

$$ = E_x \left[ {X_k^\prime X_k^{\prime*} } \right]S_{pq} E_x \left[ {X_l^\prime X_l^{\prime*} } \right] + \delta _{kl} E_x \left[ {X_k^\prime X_l^{\prime*} } \right]\sum\limits_p {S_{pp} } E_x \left[ {X_p^{\prime*} X_p^\prime } \right]$$

But δ _kl =1 if k = l

$$ = \Delta S\Delta + \Delta tr\left( {S\Delta } \right)$$

Note for $^\wedge V_{22}^\prime \left( k \right)$

From $E\left[ {H_d \left( k \right)X^\prime \left( k \right)X^\prime{H} \left( k \right)H_d^H \left( k \right)} \right]$

the k, lth element is

$$\begin{aligned} = \sum\limits_p {\sum\limits_q {E_x } } \left[ {H_{d_{k,p} } X_p^\prime X_q^{\prime*} H_{d_{q,l} }^* } \right] \\ = \sum\limits_p {\sum\limits_q E } \left[ {H_{d_{k,p} } H_{d_{q,l} }^* } \right]E\left[ {X_p^\prime X_q^{\prime*} } \right] \\ \end{aligned} $$

But $E_x \left[ {X_p^{\prime*} X_q^\prime } \right]$ is zero for all p ≠ q Since $E_x \left[ {X^\prime X^\prime{H} } \right] = \Delta$ and $E\left[ {H_{k,p} H_{q,l} } \right] = 0$ when k ≠ p, q ≠ l

$$\matrix {{\text{So,}}} { = \delta _{kl} E\left[ {H_{d_{k,k} } H_{d_{l,l} }^* } \right]E\left[ {X_k^\prime X_l^{\prime*} } \right]} \ $$

Back in the matrix form:

$$ = E\left[ {H_d \left( k \right)H_d^H \left( k \right)} \right]\Delta $$

We can let $B_1 \left( k \right) = E\left[ {H_d \left( k \right)H_d^H \left( k \right)} \right]$ instead of $E\left[ {H_d \left( k \right)} \right]$