Convergence Analysis of Möller Algorithm for Estimating Minor Component

Abstract

The minor component analysis (MCA) deals with the recovery of the eigenvector associated with the smallest eigenvalue of the autocorrelation matrix of the input data, and Möller algorithm is a famous self-stability MCA method. In this paper, we present a convergence analysis of Möller algorithm for estimating minor component of an input signal via a deterministic discrete time method. Some sufficient conditions are obtained to guarantee the convergence of Möller algorithm. Simulations are carried out to further illustrate the theoretical results achieved.

Appendices

Appendix 1: Proof of Theorem 1

Proof

By using (6)–(8), we have

$$\begin{aligned} \left\| {{\varvec{w}}(k+1)} \right\| ^{2}&= \sum _{i=1}^n {z_i^{2}(k+1)} \nonumber \\&= \sum _{i=1}^n {\left\{ {1-\eta [\lambda _i (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)-{\varvec{w}}^{T}(k){\varvec{Rw}}(k)]} \right\} ^{2}z_i^{2}(k)} \nonumber \\&\le \sum _{i=1}^n {\left[ {1-\lambda _i \eta (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)+\eta \lambda _1 \left\| {{\varvec{w}}(k)} \right\| ^{2}} \right] ^{2}z_i^{2}(k)} \nonumber \\&< (1+\eta \lambda _1 +\eta \lambda _1 \left\| {{\varvec{w}}(k)} \right\| ^{2})^{2}\sum _{i=1}^n {\hbox {z}_{\mathrm{i}}^{2}(k)} \nonumber \\&= (1+\eta \lambda _1 +\eta \lambda _1 \left\| {{\varvec{w}}(k)} \right\| ^{2})^{2}\left\| {{\varvec{w}}(k)} \right\| ^{2} \end{aligned}$$

(15)

So, we obtain that

$$\begin{aligned} \left\| {{\varvec{w}}(k+1)} \right\| ^{2}<(1+\eta \lambda _1 +\eta \lambda _1 \left\| {{\varvec{w}}(k)} \right\| ^{2})^{2}\left\| {{\varvec{w}}(k)} \right\| ^{2} \end{aligned}$$

(16)

Define a differential function $f(s)=(1+\eta \lambda _1 +\eta \lambda _1 s)^{2}s$ on the interval $[0,1]$, where $s=\left\| {{\varvec{w}}(k)} \right\| ^{2}$ and $f(s)=\left\| {{\varvec{w}}(k+1)} \right\| ^{2}$, it follows that

$$\begin{aligned} \dot{f}(s)=(1+\eta \lambda _1 +\eta \lambda _1 s)(1+\eta \lambda _1 +3\eta \lambda _1 s) \end{aligned}$$

(17)

for all $0\le s\le 1$. Clearly, the roots of the function $\dot{f}(s)=0$ are

$$\begin{aligned} s_1 =-(1+\eta \lambda _1 )/\eta \lambda _1 \hbox { and }s_2 =-(1+\eta \lambda _1 )/3\eta \lambda _1 \end{aligned}$$

(18)

By using $\eta >0$ and $\lambda _1 >0$, we have that $s_1 <s_2 <0$. So it holds that $\dot{f}(s)>0$ for all $0\le s\le 1$, i.e. $f(s)$ is monotone increasing on the interval $[0,1]$. Thus, for all $0\le s\le 1$, it follows that

$$\begin{aligned} f(s)\le f(1)<(1+2\eta \lambda _1 )^{2} \end{aligned}$$

(19)

So, we have $\left\| {{\varvec{w}}(k)} \right\| <1+2\eta \lambda _1$, for all $k\ge 0$. This completes the proof of Theorem 1.

Appendix 2: Proof of Theorem 2

Proof

By using (6)–(8), we have

$$\begin{aligned} \left\| {{\varvec{w}}(k+1)} \right\| ^{2}&= \sum _{i=1}^n {\left\{ {1-\eta \lambda _i (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)+\eta {\varvec{w}}^{T}(k){\varvec{Rw}}(k)} \right\} ^{2}z_i^{2}(k)}\nonumber \\&\ge \sum _{i=1}^n {\left\{ {1-\eta \lambda _1 (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)+\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| ^{2}} \right\} ^{2}z_i^{2}(k)} \nonumber \\&> [1-\eta \lambda _1 (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)]^{2}\sum _{i=1}^n {z_i^{2}(k)} \nonumber \\&> [1-\eta \lambda _1 (2(1+2\eta \lambda _1 )^{2}-1)]^{2}\left\| {{\varvec{w}}(k)} \right\| ^{2} \end{aligned}$$

(20)

Denote $c=\{1-\eta \lambda _1 (2(1+2\eta \lambda _1 )^{2}-1)\}$. By using condition (a), we have

$$\begin{aligned} c=\{1-\eta \lambda _1 (2(1+2\eta \lambda _1 )^{2}-1)\}>\left\{ 1-0.25 *\left[ {2 *(1+2 *0.25)^{2}-1} \right] \right\} =0.1250>0 \end{aligned}$$

(21)

From (20) and (21), we have

$$\begin{aligned} \left\| {{\varvec{w}}(k)} \right\| ^{2}>c^{2}\left\| {{\varvec{w}}(k-1)} \right\| ^{2}>\cdots >c^{2k}\left\| {{\varvec{w}}(0)} \right\| ^{2} \end{aligned}$$

(22)

So, we have$\left\| {{\varvec{w}}(k)} \right\| >c^{k}\left\| {{\varvec{w}}(0)} \right\|$, for all $k\ge 0$. This completes the proof of Theorem 2.

Appendix 3: Proof of Lemma 1

Proof

By using (6)–(8), we have

$$\begin{aligned}&1-\eta \lambda _i (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)+\eta {\varvec{w}}^{T}(k){\varvec{Rw}}(k) \nonumber \\&\quad >1-\eta \lambda _1 (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)+\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| ^{2} \nonumber \\&\quad >1-\eta \lambda _1 (2(1+2\eta \lambda _1 )^{2}-1) \nonumber \\&\quad >1-0.25 *(2 *(1+2 *0.25)^{2}-1) \nonumber \\&\quad =0.1250>0 \end{aligned}$$

(23)

This completes the proof of Lemma 1.

Appendix 4: Proof of Lemma 2

Proof

By using (6)–(8), we have for all $k\ge 0$ and $i=1,2,\ldots , n$ that

$$\begin{aligned} 1-\eta \lambda _i (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)+\eta {\varvec{w}}^{T}(k){\varvec{Rw}}(k)>0 \end{aligned}$$

(24)

By using Theorem 1, it holds that $\left\| {{\varvec{w}}(k)} \right\| <1+2\eta \lambda _1$ for all $k\ge 0$, then we have

$$\begin{aligned} \left[ {\frac{z_i (k+1)}{z_n (k+1)}} \right] ^{2}&= \left\{ {\frac{1-\eta \left[ {\lambda _i (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)-{\varvec{w}}(k)^{T}{\varvec{Rw}}(k)} \right] }{1-\eta \left[ {\lambda _n (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)-{\varvec{w}}(k)^{T}{\varvec{Rw}}(k)} \right] }} \right\} ^{2}\left[ {\frac{z_i (k)}{z_n (k)}} \right] ^{2} \nonumber \\&= \left\{ {1-\frac{\eta (\lambda _i -\lambda _n )(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)-\eta \lambda _n (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}} \right\} ^{2}\left[ {\frac{z_i (k)}{z_n (k)}} \right] ^{2} \nonumber \\&= \left\{ {1-\frac{\eta (\lambda _i -\lambda _n )}{\frac{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)}{(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}-\eta \lambda _n }} \right\} ^{2}\left[ {\frac{z_i (k)}{z_n (k)}} \right] ^{2}. \end{aligned}$$

(25)

Denote $\beta _k =\left\{ {1-\frac{\eta (\lambda _i -\lambda _n )(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)-\eta \lambda _n (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}} \right\} ^{2}$, then we will prove that $\beta _k <1$, which can be completed by the following two equations. To make the proof more brief, let’s denote ${\beta }'_k =\frac{\eta (\lambda _i -\lambda _n )(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)-\eta \lambda _n (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}$, then we have $\beta _k =(1-{\beta }'_k )^{2}$ and

$$\begin{aligned} {\beta }_k^{\prime }&= \frac{\eta (\lambda _i -\lambda _n )(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)-\eta \lambda _n (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)} \nonumber \\&< \frac{\eta \lambda _1 (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}{1+\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| ^{2}-\eta \lambda _n (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)} \nonumber \\&= \frac{\eta \lambda _1 (2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}{1-\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| ^{2}+\eta \lambda _n } \nonumber \\&< \frac{\eta \lambda _1 \left[ {2(1+2\eta \lambda _1 )^{2}-1} \right] }{1-\eta \lambda _1 (1+2\eta \lambda _1 )^{2}} \nonumber \\&< \frac{0.25\times [2\times (1+2\times 0.25)^{2}-1]}{1-0.25\times (1+2\times 0.25)^{2}} \nonumber \\&= 2. \end{aligned}$$

(26)

Denote ${\beta }_k^{\prime \prime } =\frac{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)}{(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}-\eta \lambda _n$, then ${\beta }_k^{\prime } =\eta (\lambda _i -\lambda _n ){\beta }_k^{\prime \prime }$. By using condition (a), we have

$$\begin{aligned} {\beta }_k^{\prime \prime }&= \frac{1+\eta {\varvec{w}}(k)^{T}{\varvec{Rw}}(k)}{(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}-\eta \lambda _n \nonumber \\&> \frac{1+\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| ^{2}}{(2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1)}-\eta \lambda _n \nonumber \\&= \frac{1+\eta \lambda _n -\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| ^{2}}{2\left\| {{\varvec{w}}(k)} \right\| ^{2}-1} \nonumber \\&> \frac{1-\eta \lambda _1 (1+2\eta \lambda _1 )^{2}}{2(1+2\eta \lambda _1 )^{2}-1} \nonumber \\&> \frac{1-0.25\times (1+2\times 0.25)^{2}}{2\times (1+2\times 0.25)^{2}-1} \nonumber \\&= 0.125>0 \end{aligned}$$

(27)

By using $\lambda _i -\lambda _n >0$ and (27), we have $0<{\beta }_k^{'} <2$, that is to say, $\beta _k =(1-{\beta }_k^{'})^{2}<1$.

Denote $\beta =\max (\beta _0, \beta _1, \ldots , \beta _k, \ldots )$ and $\theta _1 =\,{-}\ln \beta$, then we have $\theta _1 >0$ and

$$\begin{aligned} \left[ {\frac{z_j (k+1)}{z_n (k+1)}} \right] ^{2}&= \beta _k \left[ {\frac{z_j (k)}{z_n (k)}} \right] ^{2}=\cdots =\beta _k \beta _{k-1} \beta _0 \left[ {\frac{z_j (0)}{z_n (0)}} \right] ^{2}\le \beta ^{k+1}\left[ {\frac{z_j (0)}{z_n (0)}} \right] ^{2}\nonumber \\&= \frac{z_j^{2}(0)}{z_n^{2}(0)}e^{-\theta _1 (k+1)} \end{aligned}$$

(28)

By using $\theta _1 >0$ and (28), we have for all $i=1,2,\ldots , n-1$ that

$$\begin{aligned} \mathop {\lim }\limits _{k\rightarrow \infty } \frac{z_i (k)}{z_n (k)}=0 \end{aligned}$$

(29)

From Theorems 1 and 2, we obtain $z_n (k)$ must be bounded, then $\mathop {\lim }\nolimits _{k\rightarrow \infty } z_i (k)=0,(i=1,2,\ldots , n-1)$. This completes the proof of Lemma 2.

Appendix 5: Proof of Lemma 3

Proof

By using Lemma 2, we obtain that ${\varvec{w}}(k)$ will converge to the direction of the minor component${\varvec{v}}_{{\varvec{n}}}$, as $k\rightarrow \infty$. Suppose ${\varvec{w}}(k)$ has converged to the direction of ${\varvec{v}}_{{\varvec{n}}}$ at time $k_0$, i.e. ${\varvec{w}}(k_0 )=z_n (k_0 ){\varvec{v}}_{{\varvec{n}}}$.

From (8), we have

$$\begin{aligned} z_n (k+1)&= (1-\eta \lambda _n (2z_n^{2}(k)-1)+\eta \lambda _n z_n^{2}(k))z_n (k) \nonumber \\&= (1-2\eta \lambda _n z_n^{2}(k)+\eta \lambda _n +\eta \lambda _n z_n^{2}(k))z_n (k) \nonumber \\&= [1+\eta \lambda _n (1-z_n^{2}(k))]z_n (k) \end{aligned}$$

(30)

By using (30), we have for all $k>k_0$ that

$$\begin{aligned} z_n (k+1)-1&= [1+\eta \lambda _n (1-z_n^{2}(k))]z_n (k)-1 \nonumber \\&= (z_n (k)-1)\left[ {1-\eta \lambda _n z_n (k)(z_n (k)+1)} \right] \end{aligned}$$

(31)

By using $\left\| {{\varvec{w}}(k)} \right\| <1+2\eta \lambda _1$, we have for all $k>k_0$ that

$$\begin{aligned} 1-\eta \lambda _n z_n (k)(z_n (k)+1)&> 1-\eta \lambda _n \left\| {{\varvec{w}}(k)} \right\| (\left\| {{\varvec{w}}(k)} \right\| +1) \nonumber \\&> 1-\eta \lambda _1 \left\| {{\varvec{w}}(k)} \right\| (\left\| {{\varvec{w}}(k)} \right\| +1) \nonumber \\&> 1-\eta \lambda _1 (1+2\eta \lambda _1 )(1+2\eta \lambda _1 +1) \nonumber \\&> 1-0.25 *(1+2 *0.25) *(2+2 *0.25) \nonumber \\&= 0.0625>0 \end{aligned}$$

(32)

Denote $\delta _k =1-\eta \lambda _n z_n (k)(z_n (k)+1)$, since $z_n (k)>0$, then we can conclude that $0<\delta _k <1$. By using (30)–(32), we have for all $k>k_0$ that

$$\begin{aligned} \left| {z_n (k+1)-1} \right| =\delta _k \left| {z_n (k)-1} \right| =\delta _{k-1} \delta _k \left| {z_n (k-1)-1} \right| =\cdots =\prod _{r=0}^k {\delta _r } \left| {z_n (0)-1} \right| \end{aligned}$$

(33)

Denote $\delta =\max (\delta _k, \delta _{k-1}, \ldots , \delta _0 )$, clearly $0<\delta <1$. By using (33), we have for all $k>k_0$ that

$$\begin{aligned} \left| {z_n (k+1)-1} \right| \le \delta ^{k+1}\left| {z_n (0)-1} \right| \le (k+1)\prod e^{-\theta (k+1)} \end{aligned}$$

(34)

where $\theta ={-}\ln \delta , \prod =\left| {z_n (0)-1} \right|$.

Denote $\prod _2 =\eta \lambda _1 \left( {1+\eta \lambda _1 } \right) \left( {2+\eta \lambda _1 } \right) \left| {z_n (0)-1} \right|$. Given any $\varepsilon >0$, there exists a $K\ge 1$ such that

$$\begin{aligned} \frac{\prod _2 Ke^{-\theta K}}{(1-e^{-\theta })}\le \varepsilon \end{aligned}$$

(35)

For any $k_1 >k_2 >K$, it follows from (34) that:

$$\begin{aligned}&\left| {z_n (k_1 )-z_n (k_2 )} \right| \nonumber \\&= \left| {\sum _{r=k_2 }^{k_1 -1} {\left[ {z_n (r+1)-z_n (r)} \right] } } \right| \nonumber \\&= \left| {\sum _{r=k_2 }^{k_1 -1} {[\eta \lambda _n z_n (r)(1-z_n^{2}(r))]} } \right| \nonumber \\&\le \sum _{r=k_2 }^{k_1 -1} {\left| {\eta \lambda _n z_n (r)(1-z_n^{2}(r))} \right| } \nonumber \\&= \sum _{r=k_2 }^{k_1 -1} {\left| {\eta \lambda _n z_n (r)(z_n (r)+1)(z_n (r)-1)} \right| } \nonumber \\&<\eta \lambda _1 (1+2\eta \lambda _1 )(1+2\eta \lambda _1 +1)\sum _{r=k_2 }^{k_1 -1} {\left| {z_n (r)-1} \right| } \le \prod _2 \sum _{r=k_2 }^{k_1 -1} {re^{-\theta r}} \nonumber \\&\le \prod _2 \sum _{r=\hbox {K}}^{k_1 -1} {re^{-\theta r}}\nonumber \\&\le \prod _2 \hbox {K}e^{-\theta K}\sum _{r=0}^{+\infty } {r(e^{-\theta })^{r-1}} \nonumber \\&\le \frac{\prod _2 \hbox {K}e^{-\theta K}}{\left( {1-e^{-\theta }} \right) ^{2}} \nonumber \\&\le \varepsilon \end{aligned}$$

(36)

This implies that the sequence $\left\{ {z_n (k)} \right\}$ is a Cauchy sequence. By using the Cauchy convergence principle, there must exist a constant $z^{{*}}$ such that $\mathop {\lim }\nolimits _{k\rightarrow \infty } z_n (k)=z^{{*}}$.

By using (7), we have that $\mathop {\lim }\nolimits _{k\rightarrow \infty } {\varvec{w}}(k)=z^{{*}}{\varvec{v}}_{{\varvec{n}}}$. Since (6) has self-stabilizing property, we have that $\mathop {\lim }\nolimits _{k\rightarrow \infty } {{\varvec{w}}(k+1)}/{{\varvec{w}}(k)}=1$. From (8), we have that $z^{{*}}=\{1-\eta \lambda _n [2(z^{{*}})^{2}-1]+\eta \lambda _n (z^{{*}})^{2}\}z^{{*}}$, then we obtain that $z^{{*}}=\pm 1$. This completes the proof of Lemma 3.