Smoothing of adaptive eigenvector extraction in nested orthogonal complement structure with minimum disturbance principle

Abstract

For adaptive extraction of generalized eigensubspace, Nguyen, Takahashi and Yamada proposed a scheme for solving generalized Hermitian eigenvalue problem based on nested orthogonal complement structure. This scheme can extract multiple generalized eigenvectors by combining with any algorithm designed for estimation of the first minor generalized eigenvector. In this paper, we carefully analyse the effect of a discontinuous function employed in the scheme, and show that the discontinuous function can cause unsmooth changes of the estimates by the scheme in its adaptive implementation. To remedy the weakness, we newly introduce a projection step, for smoothing, without increasing the order of the computational complexity. Numerical experiments show that the learning curves of the non-first generalized eigenvectors are improved drastically through the proposed smoothing even when the original scheme results in unexpected performance degradation.

Appendices

Appendix 1: Inversions of matrices $\varvec{B}^{(i)}$ and $\varvec{A}^{(i)}$

Proposition A

The matrix $\varvec{Q}^{(i+1)}_{\varvec{B}}:= (\varvec{B}^{(i+1)})^{-1}$ can be expressed in terms of $\varvec{Q}^{(i)}_{\varvec{B}}(k) := (\varvec{B}^{(i)}(k))^{-1}$ as

$$\begin{aligned} \varvec{Q}_{\varvec{B}}^{(i+1)} = \left( \varvec{U}_{\perp }^{(i)} \right) ^H \varvec{Q}_{\varvec{B}}^{(i)} \varvec{U}_{\perp }^{(i)} -\left( \varvec{U}_{\perp }^{(i)} \right) ^H \varvec{u}_1^{(i)} \left( \varvec{u}_1^{(i)}\right) ^H \varvec{U}_{\perp }^{(i)}. \end{aligned}$$

(43)

Similarly, $\varvec{Q}^{(i+1)}_{\varvec{A}}(k) := (\varvec{A}^{(i+1)})^{-1}$ can be expressed in terms of $\varvec{Q}^{(i)}_{\varvec{A}}:=(\varvec{A}^{(i)})^{-1}$ as

$$\begin{aligned} \varvec{Q}_{\varvec{A}}^{(i+1)}&= \left( \varvec{U}_{\perp }^{(i)}\right) ^H \varvec{Q}_{\varvec{A}}^{(i)} \varvec{U}_{\perp }^{(i)} -\left( \varvec{U}_{\perp }^{(i)} \right) ^H \frac{\varvec{Q}_{\varvec{A}}^{(i)}\bar{\varvec{u}}_1^{(i)} \left( \varvec{Q}_{\varvec{A}}^{(i)}\bar{\varvec{u}}_1^{(i)}\right) ^H }{\left( \bar{\varvec{u}}_1^{(i)}\right) ^H\varvec{Q}_{\varvec{A}}^{(i)}\bar{\varvec{u}}_1^{(i)}} \varvec{U}_{\perp }^{(i)}. \end{aligned}$$

(44)

In the following, we show the proof of (44). For a proof of (43), see Corollary 2 in Nguyen et al. (2013).

Proof

Since $\varvec{U}^{(i)} := \begin{bmatrix} \bar{\varvec{u}}_1^{(i)}&\varvec{U}^{(i)}_{\perp } \end{bmatrix}$ is a unitary matrix from the definition of the $\varvec{B}^{(i)}$-orthogonal complement matrix (see (7)), $\bar{\varvec{u}}_1^{(i)}\in {\mathbb {C}}^{N-i+1}$ and $\varvec{U}^{(i)}_{\perp } \in {\mathbb {C}}^{(N-i+1)\times (N-i)}$ satisfy

$$\begin{aligned} \bar{\varvec{u}}_1^{(i)} \left( \bar{\varvec{u}}_1^{(i)}\right) ^H + \varvec{U}^{(i)}_{\perp }\left( \varvec{U}^{(i)}_{\perp }\right) ^H = \varvec{I}_{N-i+1} . \end{aligned}$$

(45)

Then,

$$\begin{aligned}&\varvec{A}^{(i+1)}\varvec{Q}^{(i+1)}_{\varvec{A}}\\&\quad =(\varvec{U}^{(i)})^{H}\varvec{A}^{(i)}\varvec{U}^{(i)}(\varvec{U}^{(i)})^{H}\left( \varvec{I}_{N-i+1}-\frac{\varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}(\bar{\varvec{u}}_1^{(i)})^H}{(\bar{\varvec{u}}_1^{(i)})^H \varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}}\right) \varvec{Q}_A^{(i)}\varvec{U}^{(i)} \\&\quad =(\varvec{U}^{(i)})^{H}\varvec{A}^{(i)}\left( \varvec{I}_{N-i+1}-\bar{\varvec{u}}_1^{(i)}(\bar{\varvec{u}}_1^{(i)})^H\right) \left( \varvec{I}_{N-i+1}-\frac{\varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}(\bar{\varvec{u}}_1^{(i)})^H}{(\bar{\varvec{u}}_1^{(i)})^H \varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}}\right) \varvec{Q}_A^{(i)}\varvec{U}^{(i)} \\&\qquad \qquad \qquad \quad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\mathrm {from} \,\, (45)) \\&\quad =(\varvec{U}^{(i)})^{H}\varvec{A}^{(i)}\left( \varvec{I}_{N-i+1}-\frac{\varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}(\bar{\varvec{u}}_1^{(i)})^H}{(\bar{\varvec{u}}_1^{(i)})^H \varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}}\right) \varvec{Q}_A^{(i)}\varvec{U}^{(i)} \\&\quad =\varvec{I}_{N-i} - \frac{(\varvec{U}^{(i)})^{H}\bar{\varvec{u}}_1^{(i)}(\bar{\varvec{u}}_1^{(i)})^H\varvec{Q}_A^{(i)}\varvec{U}^{(i)}}{(\bar{\varvec{u}}_1^{(i)})^H \varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}} \\&\qquad \qquad \qquad (\, \varvec{A}^{(i)} \varvec{Q}_A^{(i)} = \varvec{I}_{N-i+1}\quad \mathrm {and} \quad (\varvec{U}^{(i)})^H \varvec{U}^{(i)}=\varvec{I}_{N-i} \quad (\mathrm {see} \,\, (7)) \,) \\&\quad =\varvec{I}_{N-i} - \frac{(\varvec{U}^{(i)})^{H}\left( \varvec{I}_{N-i+1} - \varvec{U}^{(i)}(\varvec{U}^{(i)})^H\right) \varvec{Q}_A^{(i)}\varvec{U}^{(i)}}{(\bar{\varvec{u}}_1^{(i)})^H \varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}}\\&\qquad \qquad \qquad \quad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\mathrm {from} \,\, (45)) \\&\quad =\varvec{I}_{N-i} - \frac{(\varvec{U}^{(i)})^{H}\varvec{Q}_A^{(i)}\varvec{U}^{(i)} - (\varvec{U}^{(i)})^{H}\varvec{U}^{(i)}(\varvec{U}^{(i)})^{H}\varvec{Q}_A^{(i)}\varvec{U}^{(i)} }{(\bar{\varvec{u}}_1^{(i)})^H \varvec{Q}_A^{(i)}\bar{\varvec{u}}_1^{(i)}}\\&\quad =\varvec{I}_{N-i}. \end{aligned}$$

$\square$

Appendix 2: Proof of Lemma 1

Since Lemma 1(a) is obvious, we focus on the proof of Lemma 1(b). To show that $\phi _N$ is discontinuous at any point $\varvec{w}_0 := (\varvec{w}_{\mathrm{up}}^T, 0)^T \in {\mathcal {S}}_N \cap ({\mathbb {C}}^{N-1} \times \{0\})$ (note that $\Vert \varvec{w}_{\mathrm{up}}\Vert =1$), we show that for $\epsilon _0 := 1$, $\forall \delta > 0, \exists \varvec{w}$ s.t. $\Vert \varvec{w} - \varvec{w}_0\Vert < \delta$ and $\Vert \phi _N(\varvec{w})-\phi _N(\varvec{w}_0)\Vert _F \ge \epsilon _0$, where $\Vert \cdot \Vert _F$ stands for the Frobenius norm. For arbitrary fixed $\delta >0$, define $\varvec{w}(\delta ) \in {\mathcal {S}}_N$ as

$$\begin{aligned} \varvec{w}(\delta ) := \begin{pmatrix} \sqrt{1- w_{\mathrm{low}}^2} \varvec{w}_{\mathrm{up}} \\ -w_{\mathrm{low}} \end{pmatrix},\quad w_{\mathrm{low}}:= {\left\{ \begin{array}{ll} 1/2 ,&{} {\mathrm{if \ } } \delta >1 \mathrm{;}\\ \delta /2 ,&{} \mathrm{otherwise.} \end{array}\right. } \end{aligned}$$

(46)

Below, we show that $\varvec{w}(\delta )$ satisfies $\Vert \varvec{w}(\delta ) - \varvec{w}_0\Vert < \delta$ and $\Vert \phi _N(\varvec{w}(\delta ))-\phi _N(\varvec{w}_0)\Vert _F > \epsilon _0$. At first,

$$\begin{aligned} \Vert \varvec{w}(\delta )-\varvec{w}_0\Vert&= \left\| \begin{pmatrix} \left( \sqrt{1- w_{\mathrm{low}}^2} - 1\right) \varvec{w}_{\mathrm{up}} \\ -w_{\mathrm{low}} \end{pmatrix}\right\| \\&= \sqrt{(1-\sqrt{1- w_{\mathrm{low}}^2})^2\Vert \varvec{w}_{\mathrm{up}}\Vert ^2 + w_{\mathrm{low}} ^2}\\&=\sqrt{2(1-\sqrt{1-w_{\mathrm{low}}^2})}\\&<\sqrt{2(1-(1-w_{\mathrm{low}}^2))}\ \ (\because 0<1-w_{\mathrm{low}}^2<1)\\&=\sqrt{2}w_{\mathrm{low}} = {\left\{ \begin{array}{ll} \sqrt{2}/2 ,&{} {\mathrm{if \ } } \delta >1 \mathrm{;}\\ \sqrt{2}\delta /2 ,&{} \mathrm{otherwise.} \end{array}\right. } \\&<\delta \end{aligned}$$

Next,

$$\begin{aligned} \Vert \phi _N(\varvec{w}(\delta )) - \phi _N(\varvec{w}_0)\Vert _F&= \left\| \begin{bmatrix} w_{\mathrm{low}}\varvec{w}_{\mathrm{up}}\varvec{w}_{\mathrm{up}}^H \\ (-\theta (0)+\theta (-w_{\mathrm{low}})\sqrt{1- w_{\mathrm{low}}^2})\varvec{w}_{\mathrm{up}}^H \end{bmatrix}\right\| _F \\&>\left\| \begin{bmatrix} O_{(N-1) \times (N-1)} \\ (-1-\sqrt{1- w_{\mathrm{low}}^2})\varvec{w}_{\mathrm{up}}^H \end{bmatrix}\right\| _F \\&= 1+\sqrt{1- w_{\mathrm{low}}^2} > 1 = \epsilon _0. \end{aligned}$$

From the above discussion, $\forall \delta >0$, $\exists \varvec{w}$ s.t. $\Vert \varvec{w} - \varvec{w}_0\Vert < \delta$ and $\Vert \phi _N(\varvec{w}) - \phi _N(\varvec{w}_0)\Vert _F \ge \epsilon _0$. Thus the mapping $\phi _N$ is discontinuous at $\varvec{w}_0$. $\square$

Appendix 3: Proof of Proposition 1

• (i) The condition for the column vectors of ${\varvec{\perp }^{{\mathcal {I}}}_i}(k)$ to form a standard orthonormal basis of (span$(\{\varvec{u}_s^{(1)}(k)\}_{s=1}^{i-1})^{\perp }_{\langle \varvec{B} \rangle }$ is equivalent to

  $$\begin{aligned} ({\varvec{\perp }^{{\mathcal {I}}}_i}(k))^H {\varvec{\perp }}^{{\mathcal {I}}}_i(k) =\varvec{I}_{N-i+1} \end{aligned}$$

  (47)

  and

  $$\begin{aligned} ({\varvec{\perp }^{{\mathcal {I}}}_i}(k))^H\varvec{B} \varvec{u}_j^{(1)}(k) = 0 \quad \quad (j = 1,\ldots ,i-1). \end{aligned}$$

  (48)

  The relation (47) is verified as

  $$\begin{aligned} ({\varvec{\perp }^{{\mathcal {I}}}_i}(k))^H {\varvec{\perp }^{{\mathcal {I}}}_i}(k)&= \left( \prod _{s=1}^{i-1} \varvec{U}_{\perp }^{(s)}(k)\right) ^H \left( \prod _{s=1}^{i-1} \varvec{U}_{\perp }^{(s)}(k)\right) \\&=\left( \varvec{U}_{\perp }^{(i-1)}(k)\right) ^H \cdots \left( \varvec{U}_{\perp }^{(1)}(k)\right) ^H \left( \varvec{U}_{\perp }^{(1)}(k)\right) \cdots \left( \varvec{U}_{\perp }^{(i-1)}(k)\right) \\&= \varvec{I}_{N-i+1} \quad \quad \left( \mathrm {From}\,(7), \, \left( \varvec{U}_{\perp }^{(s)}(k)\right) ^H \varvec{U}_{\perp }^{(s)}(k) = \varvec{I}_{N-s}\right) . \end{aligned}$$

  The relation (49) is verified as, for $j=1,\ldots ,i-1$,

  $$\begin{aligned}&({\varvec{\perp }^{{\mathcal {I}}}_i}(k))^H\varvec{B} \varvec{u}_j^{(1)}(k) \\&\quad = \left( \prod _{s=1}^{i-1} \varvec{U}_{\perp }^{(s)}(k)\right) ^H \varvec{B} \left( \prod _{s=1}^{j-1} \varvec{U}_{\perp }^{(s)}\right) \varvec{u}_1^{(j)} (k)\\&\quad = \left( \varvec{U}_{\perp }^{(i-1)}(k)\right) ^H \cdots \left( \varvec{U}_{\perp }^{(1)}(k)\right) ^H \varvec{B} \left( \varvec{U}_{\perp }^{(1)}(k)\right) \cdots \left( \varvec{U}_{\perp }^{(j-1)}(k)\right) \varvec{u}_1^{(j)} (k)\\&\quad = \left( \varvec{U}_{\perp }^{(i-1)}(k)\right) ^H \cdots \left( \varvec{U}_{\perp }^{(j)}(k)\right) ^H \varvec{B}^{(j)}(k) \varvec{u}_1^{(j)}(k)\quad \left( \mathrm {From}\, (25) \right) \\&\quad =\varvec{0} \end{aligned}$$

  where the last equality holds by $\varvec{U}^{(j)}_{\perp }(k) = \varvec{U}_{\perp [\varvec{B}^{(j)}(k)]}(\varvec{u}_1^{(j)}(k))$ and $\left( \varvec{U}_{\perp }^{(j)}(k)\right) ^H \varvec{B}^{(j)}(k) \varvec{u}_1^{(j)}(k) = \varvec{0}$ (see (7), for the property of $\varvec{B}^{(j)}(k)$-orthogonal complement matrix).

• (ii) The projection of $\varvec{z} \in {\mathbb {C}}^{N}$ onto ${\mathcal {R}}({\varvec{\perp }^{{\mathcal {I}}}_i}(k))$ is

  $$\begin{aligned} P_{{\mathcal {R}}({\varvec{\perp }^{{\mathcal {I}}}_i}(k))}(\varvec{z})&= {\varvec{\perp }^{{\mathcal {I}}}_i}(k)\left( ({\varvec{\perp }^{{\mathcal {I}}}_i}(k))^H {\varvec{\perp }}^{{\mathcal {I}}}_i(k) \right) ^{-1}\left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{z} \nonumber \\&={\varvec{\perp }^{{\mathcal {I}}}_i}(k)\left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{z} \quad \mathrm {( From \, (47) )}, \end{aligned}$$

  (49)

  where the first equality holds from the Normal equation [see Sect 3.6 in Luenberger (1969)].$\square$

Appendix 4: Proof of Proposition 2

From (25),

$$\begin{aligned}&\Vert {\varvec{\perp }^{{\mathcal {I}}}_i}(k) ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)\Vert _{\varvec{B}}\\&\quad = \sqrt{ \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k)( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)\right) ^H \varvec{B}{\varvec{\perp }^{{\mathcal {I}}}_i}(k) ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)}\\&\quad = \sqrt{ \left( ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)\right) ^H ({\varvec{\perp }^{{\mathcal {I}}}_i}(k))^H \varvec{B}{\varvec{\perp }^{{\mathcal {I}}}_i}(k) ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)}\\&\quad = \sqrt{ \left( ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)\right) ^H \varvec{B}^{(i)}(k) ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)}\\&\quad =\Vert ( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)\Vert _{\varvec{B}^{(i)}(k)} . \end{aligned}$$

Therefore,

$$\begin{aligned} \widetilde{\varvec{u}}^{(i)}_1(k-1) := \frac{ \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1) }{ \Vert {\varvec{\perp }^{{\mathcal {I}}}_i}(k) ({\varvec{\perp }^{{\mathcal {I}}}_i}(k) )^H \varvec{u}_i^{(1)}(k-1)\Vert _{\varvec{B}}}= \frac{ \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1) }{ \Vert \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1)\Vert _{\varvec{B}^{(i)}(k)} }. \end{aligned}$$

$\square$

Appendix 5: Computational complexity of steps in Scheme 2I

In this section, we evaluate the total computational complexity of Step 2b, Step 2c and Step 3 of Scheme 2I at time k. We first consider computational complexity of Step 2b and 2c for fixed i. In the following, we omit k for simplicity.

In Step 2b, we have to calculate $\varvec{B}^{(i)}\varvec{u}_1^{(i)}$ to obtain $\varvec{U}_{\perp }^{(i)}$. Therefore, Step 2b requires $(N-i+1)^2 + {\mathcal {O}} (N-i)$ multiplications. In Step 2c, we have to calculate $\left( \varvec{U}_{\perp }^{(i)}\right) ^H\varvec{A}^{(i)} \varvec{U}_{\perp }^{(i)}$ and $\left( \varvec{U}_{\perp }^{(i)}\right) ^H\varvec{B}^{(i)} \varvec{U}_{\perp }^{(i)}$. For a Hermitian matrix

$$\begin{aligned} \varvec{T} = \begin{bmatrix} \varvec{T}'&\varvec{t} \\ \varvec{t}^{H}&t \end{bmatrix}\text{, } \end{aligned}$$

$\left( \varvec{U}^{(i)}_{\perp }\right) ^H \varvec{T} \varvec{U}^{(i)}_{\perp }$, is calculated as

$$\begin{aligned}&\left( \varvec{U}^{(i)}_{\perp }\right) ^H \varvec{T} \varvec{U}^{(i)}_{\perp } \nonumber \\&\quad =\begin{bmatrix}\varvec{I}_{N-i}-a\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H , -b\bar{\varvec{u}}^{(s)}_{\mathrm{up}}\end{bmatrix} \begin{bmatrix} \varvec{T}'&\varvec{t} \nonumber \\ \varvec{t}^{H}&t \end{bmatrix}\begin{bmatrix}\varvec{I}_{N-i}-a\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H \nonumber \\ -b\left( \bar{\varvec{u}}^{(s)}_{\mathrm{up}}\right) ^H\end{bmatrix}\nonumber \\&\qquad \qquad \qquad \qquad \qquad (\, \mathrm {where} \quad a =1/(1 + |{\bar{u}}_{\mathrm{low}}|) \quad \mathrm {and} \quad b = \theta ({\bar{u}}^{(i)}_{\mathrm{low}})\, )\nonumber \\&\quad =\varvec{T}' -a\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H\varvec{T}' -a\varvec{T}'\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H + a^2\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H\varvec{T}'\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H\nonumber \\&\qquad +ab(\varvec{t}^H\bar{\varvec{u}}^{(i)}_{\mathrm{up}})\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H - b \varvec{t}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H + ab(\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H\varvec{t})\bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H\nonumber \\&\qquad + b^2 t \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H \nonumber \\&= \varvec{T}' - (a\varvec{T}'\bar{\varvec{u}}^{(i)}_{\mathrm{up}} + b\varvec{t})\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H - \left( (a\varvec{T}'\bar{\varvec{u}}^{(i)}_{\mathrm{up}} + b\varvec{t})\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H \right) ^H\nonumber \\&\qquad \left( a^2\left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H \varvec{T}'\bar{\varvec{u}}^{(i)}_{\mathrm{up}} + b^2t + 2ab\varvec{t}^H\bar{\varvec{u}}^{(i)}_{\mathrm{up}} \right) \bar{\varvec{u}}^{(i)}_{\mathrm{up}} \left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}\right) ^H. \end{aligned}$$

(50)

The second term of (50) requires $2(N-i)^2 + {\mathcal {O}}(N-i)$ multiplications. Since the third term (50) is the conjugate transpose of the second term, we can compute this term without additional multiplications. Moreover, since $\varvec{T}'\bar{\varvec{u}}^{(i)}_{\mathrm{up}}$ is calculated in the second term, the fourth term requires only $(N-i)^2 + {\mathcal {O}}(N-i)$ multiplications. Then, the total computational complexity of $\left( \varvec{U}^{(i)}_{\perp }\right) ^H \varvec{T} \varvec{U}^{(i)}_{\perp }$ is $3(N-i)^2 + {\mathcal {O}}(N-i)$. Therefore, Step 2c requires $6(N-i)^2 + {\mathcal {O}}(N-i)$ multiplications and for fixed i, Step 2b, and Step 2c require $7(N-i)^2 + {\mathcal {O}}(N-i)$ multiplications.

For fixed i, Step 3 requires $i-1$ times multiplications of orthogonal complement matrices and a vector, then the computational complexity is $2Ni-i^2 + {\mathcal {O}}(i)$ as mentioned in Sect. 3.4.

Table 1 Computational complexities of each steps of Scheme 2I at time k

Since Step 2b and 2c are performed for $1\le i \le r-1$, the total computational complexity of them is $7N^2r - 7Nr^2 + 7r^3/6 + {\mathcal {O}}(Nr)$. Step 3 requires $Nr^2 - r^3/6 + {\mathcal {O}}(Nr)$ when i moves 2 to r. Then, the total computational complexity of Step 2b, Step 2c and Step 3 of Scheme 2I is $7N^2r - 6Nr^2 +2r^3 + {\mathcal {O}}(Nr)$. This is much larger than additional computational complexity $Nr^2+{\mathcal {O}}(Nr)$ of Step 2a$_+$, i.e., projection step (see Table 1).

Remark 3

(Computational complexity for Step 2a) In Step 2a, for instance, Example 1(a) requires $3N^2r - 3Nr^2/2 +r^3/2 +{\mathcal {O}}(Nr)$ multiplications, and Example 1(b) requires $5N^2r - 5Nr^2/2 +5r^3/6 +{\mathcal {O}}(Nr)$ multiplications.

Appendix 6: Algorithm 1: Combinations of incremental schemes and FMGEs

Algorithm 1

Initialization step:

Set $k=0$, $\{\varvec{u}_i^{(1)}(0)\}_{i=1}^{r} \subset {\mathbb {C}}^N$ arbitrarily as an initial estimate of a basis of generalized minor subspace (note that $\{\varvec{u}_i^{(1)}(0)\}_{i=2}^{r}$ are used in the projection step of Scheme 2I and Scheme 3I) and, for [Example 1(b)], set $\mu _i(0)$ as an initial estimate of generalized eingenvalues. For $i=2,\ldots ,r$, set $\varvec{u}_1^{(i)}(0) \in {\mathbb {C}}^{N-i+1}$ arbitrarily as an initial estimate of the first minor generalized eigenvector of the matrix pencil $(\varvec{A}^{(i)}, \varvec{B}^{(i)})$. Moreover, for Scheme 3I, set the threshold ${\hat{\theta }}$.

Iteration step:

Step1 $k\leftarrow k+1$, $i = 1$ and set $\varvec{A}^{(1)}(k) = \varvec{A}$, $\varvec{B}^{(1)}(k) = \varvec{B}$ and

[Example 1(a)] $\varvec{Q}^{(1)}_{\varvec{B}}(k) := \varvec{B}^{-1}$ or [Example 1(b)] $\varvec{Q}^{(1)}_{\varvec{A}}(k) := \varvec{A}^{-1}$.

Step2 For $i = 1,\dots , r$,

$\mathbf{a}_{+}.$ :

Set $\widetilde{\varvec{u}}_1^{(i)}(k-1)$ as [Scheme 1I] $\widetilde{\varvec{u}}_1^{(i)}(k-1) = \varvec{u}_1^{(i)}(k-1)$. [Scheme 2I]

$$\begin{aligned} \widetilde{\varvec{u}}^{(i)}_1(k-1):= {\left\{ \begin{array}{ll} \varvec{u}^{(1)}_1(k-1)\text{, } &{} i=1 \text{; }\\ \dfrac{ \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1) }{ \Vert \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1)\Vert _{\varvec{B}^{(i)}(k)} } \text{, }&{} i=2,\ldots ,r\mathrm{.} \end{array}\right. } \end{aligned}$$

[Scheme 3I] If $i=1$, or, for $i\ge 2$, $\forall s$ ($1\le s \le i-1$) $\mathrm {angle}({\bar{u}}_{\mathrm{low}}^{(s)}(k), {\bar{u}}_{\mathrm{low}}^{(s)}(k-1)) \le {\hat{\theta }}$,

$$\begin{aligned} \widetilde{\varvec{u}}_1^{(i)}(k-1) = \varvec{u}_1^{(i)}(k-1), \end{aligned}$$

otherwise

$$\begin{aligned} \widetilde{\varvec{u}}^{(i)}_1(k-1) = \frac{ \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1) }{ \Vert \left( {\varvec{\perp }^{{\mathcal {I}}}_i}(k) \right) ^H \varvec{u}_i^{(1)}(k-1)\Vert _{\varvec{B}^{(i)}(k)} }. \end{aligned}$$

a. :

Perform a single update of $\mathrm {FMGE}(\varvec{A}^{(i)}(k),\varvec{B}^{(i)}(k))$ from $\widetilde{\varvec{u}}^{(i)}_1(k-1)$ and denote its outcome by $\varvec{u}^{(i)}_1(k)$. Calculate $\varvec{u}^{(i)}_1(k)$ as [Example 1(a)]

$$\begin{aligned} \widehat{\varvec{u}}_1^{(i)}(k)&= \left( \frac{1}{\tau }\varvec{I}_{N-i+1} - \varvec{Q}^{(i)}_{\varvec{B}}(k)\varvec{A}^{(i)}(k) \right) \widetilde{\varvec{u}}^{(i)}_1(k-1) \nonumber \\ \varvec{u}_1^{(i)}(k)&=\frac{\widehat{\varvec{u}}_1^{(i)}(k)}{\Vert \widehat{\varvec{u}}_1^{(i)}(k) \Vert _{\varvec{B}^{(i)}(k)}} , \end{aligned}$$

or [Example 1(b)]

$$\begin{aligned} \eta _i(k)&= {\left\{ \begin{array}{ll} \eta _1\text{, } &{} i=1\mathrm{;}\\ \dfrac{2\mu _i(k-1)\eta _1}{\mu _1(k-1)(2+\eta _1)-\mu _i(k-1)\eta _1} \text{, }&{} i=2, \ldots , r \mathrm{,} \end{array}\right. } \nonumber \\ \widehat{\varvec{u}}_1^{(i)}(k)&= \widetilde{\varvec{u}}_1^{(i)}(k-1) + \eta _i(k) \, \Bigl [ \varvec{Q}_{\varvec{A}}^{(i)}(k)\varvec{B}^{(i)}(k) \widetilde{\varvec{u}}_1^{(i)}(k-1)\mu _i(k-1) \nonumber \\ \nonumber&\quad \qquad +(\widetilde{\varvec{u}}_1^{(i)}(k-1))^H\varvec{A}^{(i)}(k)\widetilde{\varvec{u}}_1^{(i)}(k-1)\widetilde{\varvec{u}}(k-1)\mu _i^{-1}(k-1)\\ {}&\qquad -2\widetilde{\varvec{u}}_1^{(i)}(k-1) \Bigl ], \nonumber \\ \varvec{u}_1^{(i)}(k)&=\frac{\widehat{\varvec{u}}_1^{(i)}(k) }{\Vert \widehat{\varvec{u}}_1^{(i)}(k) \Vert _{\varvec{B}^{(i)}(k)}},\nonumber \\ \mu _i(k)&= (1-\gamma )\mu _i(k-1) +\gamma (\varvec{u}_1^{(i)}(k))^H\varvec{A}^{(i)}(k)\varvec{u}_1^{(i)}(k), \end{aligned}$$

(51)

where $\tau , \eta _i(k), \gamma >0$ are positive small stepsizes.

b. :

If $i \not = r$, compute the $\varvec{B}^{(i)}(k)$-orthogonal complement matrix $\varvec{U}_{\perp }^{(i)}(k)$ of $\varvec{u}_1^{(i)}(k)$ by (5) as

$$\begin{aligned} \bar{\varvec{u}}_1^{(i)}(k)&= \frac{\varvec{B}^{(i)}(k)\varvec{u}_1^{(i)}(k)}{\Vert \varvec{B}^{(i)}(k)\varvec{u}_1^{(i)}(k)\Vert },\\ \varvec{U}_{\perp }^{(i)}(k)&=\begin{pmatrix} \varvec{I}_{N-i}-\frac{1}{1+|{\bar{u}}_{\mathrm{low}}^{(i)}(k)|} \bar{\varvec{u}}^{(i)}_{\mathrm{up}}(k) \left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}(k)\right) ^H \\ -\theta \left( {\bar{u}}^{(i)}_{\mathrm{low}}(k)\right) \left( \bar{\varvec{u}}^{(i)}_{\mathrm{up}}(k)\right) ^H \end{pmatrix}, \end{aligned}$$

where $\Vert \cdot \Vert$ stands for the standard Euclidean norm, $\bar{\varvec{u}}^{(i)}_{\mathrm{up}}(k) \in {\mathbb {C}}^{N-i}$ and ${\bar{u}}^{(i)}_{\mathrm{low}}(k) \in {\mathbb {C}}$ are respectively the first $(N-i)$ components and the last component of the normalized vector $\bar{\varvec{u}}_1^{(i)}(k)$, i.e. $\bar{\varvec{u}}_1^{(i)}(k)$ = $\left( (\bar{\varvec{u}}^{(i)}_{\mathrm{up}}(k))^T, {\bar{u}}^{(i)}_{\mathrm{low}}(k)\right) ^T$ and

$$\begin{aligned} \theta ({\bar{u}}^{(i)}_{\mathrm{low}}(k)):= {\left\{ \begin{array}{ll} 1, &{} {\bar{u}}^{(i)}_{\mathrm{low}}(k) = 0;\\ \dfrac{{\bar{u}}_{\mathrm{low}}^{(i)}(k)}{|{\bar{u}}_{\mathrm{low}}^{(i)}(k)|}, &{} \mathrm{otherwise.} \end{array}\right. } \end{aligned}$$

c. :

If $i \not = r$, define $\varvec{A}^{(i+1)}(k)$ and $\varvec{B}^{(i+1)}(k)$ as

$$\begin{aligned} \varvec{A}^{(i+1)}(k)&= \left( \varvec{U}_{\perp }^{(i)}(k) \right) ^H \varvec{A}^{(i)}(k) \varvec{U}_{\perp }^{(i)}(k), \\ \varvec{B}^{(i+1)}(k)&= \left( \varvec{U}_{\perp }^{(i)}(k) \right) ^H \varvec{B}^{(i)}(k) \varvec{U}_{\perp }^{(i)}(k) , \end{aligned}$$

and for [Example 1(a)], calculate $\varvec{Q}_{\varvec{B}}^{(i+1)}(k):=(\varvec{B}^{i+1}(k))^{-1}$ as

$$\begin{aligned} \varvec{Q}_{\varvec{B}}^{(i+1)}(k)&= \left( \varvec{U}_{\perp }^{(i)}(k) \right) ^H \varvec{Q}_{\varvec{B}}^{(i)}(k) \varvec{U}_{\perp }^{(i)}(k) \\&\qquad \quad -\left( \varvec{U}_{\perp }^{(i)}(k) \right) ^H \varvec{u}_1^{(i)}(k) \left( \varvec{u}_1^{(i)}(k)\right) ^H \varvec{U}_{\perp }^{(i)}(k), \end{aligned}$$

or, for [Example 1(b)], calculate $\varvec{Q}_{\varvec{A}}^{(i+1)}(k):= (\varvec{A}^{(i+1)}(k))^{-1}$ as

$$\begin{aligned} \varvec{Q}_{\varvec{A}}^{(i+1)}(k)&= \left( \varvec{U}_{\perp }^{(i)}(k) \right) ^H \varvec{Q}_{\varvec{A}}^{(i)}(k) \varvec{U}_{\perp }^{(i)}(k) \\&\quad \qquad -\left( \varvec{U}_{\perp }^{(i)}(k) \right) ^H \frac{\varvec{Q}_{\varvec{A}}^{(i)}(k)\bar{\varvec{u}}_1^{(i)}(k) \left( \varvec{Q}_{\varvec{A}}^{(i)}(k)\bar{\varvec{u}}_1^{(i)}(k)\right) ^H }{\left( \bar{\varvec{u}}_1^{(i)}(k)\right) ^H\varvec{Q}_{\varvec{A}}^{(i)}(k)\bar{\varvec{u}}_1^{(i)}(k)} \varvec{U}_{\perp }^{(i)}(k). \end{aligned}$$

Step3 For $i =2,\cdots ,r$, update $\varvec{u}_i^{(1)}(k)$ as

$$\begin{aligned} \varvec{u}_i^{(1)}(k) = {\varvec{\perp }_i^{{\mathcal {I}}}}(k)\varvec{u}_1^{(i)}(k) = \varvec{U}_{\perp }^{(1)}(k)\cdots \varvec{U}_{\varvec{\perp }}^{(i-1)}(k)\varvec{u}_1^{(i)}(k). \end{aligned}$$

Remark 4

(Stepsizes for Example 1(b) in (51))

For simplicity, we assume that

$$\begin{aligned} \eta _1 := \frac{\mu _1}{\mu _N - \mu _1 } > 0 \end{aligned}$$

(52)

is available for numerical experiments. This $\eta _1$ satisfies the convergence condition (19) with i=1 for $(\varvec{A}^{(1)},\varvec{B}^{(1)}) = (\varvec{A},\varvec{B})$. Then,

$$\begin{aligned} \eta _i := \frac{2\mu _i\eta _1}{\mu _1(2+\eta _1)-\mu _i\eta _1} \end{aligned}$$

(53)

satisfies the convergence condition (19) for $(\varvec{A}^{(i)},\varvec{B}^{(i)})$ ($i=2,\ldots ,r$), which is verified by

$$\begin{aligned} \eta _i =\dfrac{2\mu _i\eta _1}{2\mu _1-(\mu _i-\mu _1)\eta _1} \ge \dfrac{2\mu _i\eta _1}{2\mu _1} \ge \eta _1 \quad (\mathrm {from}\quad \mu _i \ge \mu _1), \end{aligned}$$

and

$$\begin{aligned} \eta _i = \frac{2\mu _i}{\mu _1\left( \dfrac{2}{\eta _1}+1\right) -\mu _i}< \frac{2\mu _i}{\mu _1\left( \dfrac{2}{\frac{2\mu _1}{\mu _N - \mu _1}}+1\right) -\mu _i}=\frac{2\mu _i}{\mu _N - \mu _i}. \end{aligned}$$

The stepsize in (51) is designed by replacing the ith generalized eigenvalue $\mu _i$ $(i=1,\ldots ,r)$ of $(\varvec{A}^{(1)},\varvec{B}^{(1)})$ in (53) with their estimates $\mu _i(k-1)$.

Appendix 7: Algorithm 2: Combinations of adaptive schemes and FMGEs

Algorithm 2

Initialization step:

Set $k=0$, $\varvec{R}^{(1)}_y(0) = \varvec{R}^{(1)}_x(0) = \varvec{I}_N$, and for [Example 1(a)], set $\varvec{Q}^{(1)}_x(0)=\varvec{I}_N$, and, for [Example 1(b)], set $\varvec{Q}^{(1)}_y(0)=\varvec{I}_N$. Set $\{\varvec{w}_i^{(1)}(0)\}_{i=1}^{r} \subset {\mathbb {C}}^N$ arbitrarily as an initial estimate of a basis of the time-varying generalized minor subspace (note that$\{\varvec{w}_i^{(1)}(0)\}_{i=2}^{r} \subset {\mathbb {C}}^N$ are used in the projection step of Scheme 2A and Scheme 3A) and, for [Example 1(b)], set $\lambda _i(0)$ as an initial estimate of eigenvalues. For $i=2,\ldots ,r$, set $\varvec{w}_1^{(i)}(k) \in {\mathbb {C}}^{N-i+1}$ arbitrarily as an initial estimate of the first minor generalized eigenvector of the matrix pencil $(\varvec{R}^{(i)}_y, \varvec{R}^{(i)}_x)$. Moreover, for Scheme 3A, set the threshold ${\hat{\theta }}$.

Iteration step:

Step1 $k\leftarrow k+1$, $i = 1$ and update $\varvec{R}^{(1)}_y(k), \varvec{R}^{(1)}_x(k)$ by (4) and for [Example 1(a)], set $\varvec{Q}^{(1)}_x(k) := (\varvec{R}^{(1)}_x(k))^{-1}$ by the matrix inversion lemma (Hager 1989) as

$$\begin{aligned} \varvec{Q}^{(1)}_x(k) =\dfrac{1}{\alpha } \left( \varvec{Q}^{(1)}_x(k-1) - \dfrac{\varvec{Q}^{(1)}_x(k-1)\varvec{x}(k)(\varvec{x}(k))^H\varvec{Q}^{(1)}_x(k-1)}{\alpha +(\varvec{x}(k))^{H}\varvec{Q}^{(1)}_x(k-1)\varvec{x}(k)} \right) \end{aligned}$$

or, for [Example 1(b)], set $\varvec{Q}^{(1)}_y(k) := (\varvec{R}^{(1)}_y(k))^{-1}$ as

$$\begin{aligned} \varvec{Q}^{(1)}_y(k) =\dfrac{1}{\alpha } \left( \varvec{Q}^{(1)}_y(k-1) - \dfrac{\varvec{Q}^{(1)}_y(k-1)\varvec{y}(k)(\varvec{y}(k))^H\varvec{Q}^{(1)}_y(k-1)}{\alpha +(\varvec{y}(k))^{H}\varvec{Q}^{(1)}_y(k-1)\varvec{y}(k)} \right) . \end{aligned}$$

Step2 For $i = 1,\dots , r$,

$\mathbf{a}_{+}.$ :

Set $\widetilde{\varvec{w}}_1^{(i)}(k-1)$ as [Scheme 1A] $\widetilde{\varvec{w}}_1^{(i)}(k-1) = \varvec{w}_1^{(i)}(k-1)$. [Scheme 2A]

$$\begin{aligned} \widetilde{\varvec{w}}^{(i)}_1(k-1):= {\left\{ \begin{array}{ll} \varvec{w}^{(1)}_1(k-1)\text{, } &{} i=1 \text{; }\\ \dfrac{ \left( {\varvec{\perp }^{{\mathcal {A}}}_i}(k) \right) ^H \varvec{w}_i^{(1)}(k-1) }{ \Vert \left( {\varvec{\perp }^{{\mathcal {A}}}_i}(k) \right) ^H \varvec{w}_i^{(1)}(k-1)\Vert _{\varvec{R}_x^{(i)}(k)} } \text{, }&{} i=2,\ldots ,r\mathrm{.} \end{array}\right. } \end{aligned}$$

[Scheme 3A] If $i=1$, or, for $i\ge 2$, $\forall s$ ($1\le s \le i-1$) $\mathrm {angle}({\bar{w}}_{\mathrm{low}}^{(s)}(k), {\bar{w}}_{\mathrm{low}}^{(s)}(k-1)) \le {\hat{\theta }}$,

$$\begin{aligned} \widetilde{\varvec{w}}_1^{(i)}(k-1) = \varvec{w}_1^{(i)}(k-1), \end{aligned}$$

otherwise

$$\begin{aligned} \widetilde{\varvec{w}}^{(i)}_1(k-1) = \frac{ \left( {\varvec{\perp }^{{\mathcal {A}}}_i}(k) \right) ^H \varvec{w}_i^{(1)}(k-1) }{ \Vert \left( {\varvec{\perp }^{{\mathcal {A}}}_i}(k) \right) ^H \varvec{w}_i^{(1)}(k-1)\Vert _{\varvec{R}_x^{(i)}(k)} }. \end{aligned}$$

a. :

Perform one update of $\mathrm {FMGE}(\varvec{R}^{(i)}_y(k),\varvec{R}^{(i)}_x(k))$ from $\widetilde{\varvec{w}}^{(i)}_1(k-1)$ and denote its outcome by $\varvec{w}^{(i)}_1(k)$. Calculate $\varvec{w}^{(i)}_1(k)$ as [Example 1(a)]

$$\begin{aligned} \widehat{\varvec{w}}_1^{(i)}(k)&= \left( \frac{1}{\tau }\varvec{I}_{N-i+1} - \varvec{Q}^{(i)}_x(k)\varvec{R}^{(i)}_y(k) \right) \widetilde{\varvec{w}}^{(i)}_1(k-1), \\ \varvec{w}_1^{(i)}(k)&=\frac{\widehat{\varvec{w}}_1^{(i)}(k)}{\Vert \widehat{\varvec{w}}_1^{(i)}(k) \Vert _{\varvec{R}^{(i)}_x(k)}}\mathrm {,} \end{aligned}$$

or [Example 1(b)]

$$\begin{aligned} \eta _i(k)&= {\left\{ \begin{array}{ll} \eta _1\text{, } &{} i=1\mathrm{;}\\ \dfrac{2\lambda _i(k-1)\eta _1}{\lambda _1(k-1)(2+\eta _1)-\lambda _i(k-1)\eta _1} \text{, }&{} i=2, \ldots , r \mathrm{,} \end{array}\right. }\\ \widehat{\varvec{w}}_1^{(i)}(k)&= \widetilde{\varvec{w}}_1^{(i)}(k-1) + \eta _i(k) \, \Bigl [ \varvec{Q}_y^{(i)}(k)\varvec{R}^{(i)}_x(k) \widetilde{\varvec{w}}_1^{(i)}(k-1)\lambda _i(k-1) \\&\quad +(\widetilde{\varvec{w}}_1^{(i)}(k-1))^H\varvec{R}^{(i)}_y(k)\widetilde{\varvec{w}}_1^{(i)}(k-1)\widetilde{\varvec{w}}_1^{(i)}(k-1)\lambda _i^{-1}(k-1)\\&\quad -2 \widetilde{\varvec{w}}_1^{(i)}(k-1) \Bigl ], \\ \varvec{w}_1^{(i)}(k)&=\frac{\widehat{\varvec{w}}_1^{(i)}(k) }{\Vert \widehat{\varvec{w}}_1^{(i)}(k) \Vert _{\varvec{R}^{(i)}_x(k)}},\\ \lambda _i(k)&= (1-\gamma )\lambda _i(k-1) +\gamma (\varvec{w}_1^{(i)}(k))^H\varvec{R}^{(i)}_y(k)\varvec{w}_1^{(i)}(k). \end{aligned}$$

b. :

If $i \not = r$, compute an $\varvec{R}_x^{(i)}(k)$-orthogonal complement matrix $\varvec{W}_{\perp }^{(i)}(k)$ of $\varvec{w}_1^{(i)}(k)$ by (5) as

$$\begin{aligned} \bar{\varvec{w}}_1^{(i)}(k)&= \frac{\varvec{R}_x^{(i)}(k)\varvec{w}_1^{(i)}(k)}{\Vert \varvec{R}_x^{(i)}(k)\varvec{w}_1^{(i)}(k)\Vert }\\ \varvec{W}_{\perp }^{(i)}(k)&=\begin{pmatrix} \varvec{I}_{N-i}-\frac{1}{1+|{\bar{w}}_{\mathrm{low}}^{(i)}(k)|} \bar{\varvec{w}}^{(i)}_{\mathrm{up}}(k) \left( \bar{\varvec{w}}^{(i)}_{\mathrm{up}}(k)\right) ^H \\ -\theta ({\bar{w}}^{(i)}_{\mathrm{low}}(k)) \left( \bar{\varvec{w}}^{(i)}_{\mathrm{up}}(k)\right) ^H \end{pmatrix}, \end{aligned}$$

where $\Vert \cdot \Vert$ stands for the standard Euclidean norm, $\bar{\varvec{w}}^{(i)}_{\mathrm{up}}(k) \in {\mathbb {C}}^{N-i}$ and ${\bar{w}}^{(i)}_{\mathrm{low}}(k) \in {\mathbb {C}}$ are respectively the first $(N-i)$ components and the last component of the normalized vector $\bar{\varvec{w}}_1^{(i)}(k)$, i.e., $\bar{\varvec{w}}_1^{(i)}(k)$ = $\left( (\bar{\varvec{w}}^{(i)}_{\mathrm{up}}(k))^T, {\bar{w}}^{(i)}_{\mathrm{low}}(k) \right) ^T$ and

$$\begin{aligned} \theta ({\bar{w}}^{(i)}_{\mathrm{low}}(k)):= {\left\{ \begin{array}{ll} 1, &{} {\bar{w}}^{(i)}_{\mathrm{low}}(k) = 0\mathrm{;}\\ \dfrac{{\bar{w}}_{\mathrm{low}}^{(i)}(k)}{|{\bar{w}}_{\mathrm{low}}^{(i)}(k)|}, &{} \mathrm{otherwise}. \end{array}\right. } \end{aligned}$$

c. :

If $i \not = r$, define $\varvec{R}_y^{(i+1)}(k)$ and $\varvec{R}_x^{(i+1)}(k)$ as

$$\begin{aligned} \varvec{R}_y^{(i+1)}(k)&= \left( \varvec{W}_{\perp }^{(i)}(k) \right) ^H \varvec{R}_y^{(i)}(k) \varvec{W}_{\perp }^{(i)}(k), \\ \varvec{R}_x^{(i+1)}(k)&= \left( \varvec{W}_{\perp }^{(i)}(k) \right) ^H \varvec{R}_x^{(i)}(k) \varvec{W}_{\perp }^{(i)}(k), \end{aligned}$$

and [Example 1(a)] calculate $\varvec{Q}_x^{(i+1)}(k) :=(\varvec{R}^{(i+1)}_x(k))^{-1}$ for as

$$\begin{aligned} \varvec{Q}_x^{(i+1)}(k)&= \left( \varvec{W}_{\perp }^{(i)}(k) \right) ^H \varvec{Q}_x^{(i)}(k) \varvec{W}_{\perp }^{(i)}(k) \\&\qquad \quad -\left( \varvec{W}_{\perp }^{(i)}(k) \right) ^H \varvec{w}_1^{(i)}(k) \left( \varvec{w}_1^{(i)}(k)\right) ^H \varvec{W}_{\perp }^{(i)}(k), \end{aligned}$$

or, for [Example 1(b)] calculate $\varvec{Q}_y^{(i+1)}(k):=(\varvec{R}^{(i+1)}_y(k))^{-1}$ as

$$\begin{aligned} \varvec{Q}_y^{(i+1)}(k)&= \left( \varvec{W}_{\perp }^{(i)}(k) \right) ^H \varvec{Q}_y^{(i)}(k) \varvec{W}_{\perp }^{(i)}(k) \\&\qquad \quad -\left( \varvec{W}_{\perp }^{(i)}(k) \right) ^H \frac{\varvec{Q}_y^{(i)}(k)\bar{\varvec{w}}_1^{(i)}(k) \left( \varvec{Q}_y^{(i)}(k)\bar{\varvec{w}}_1^{(i)}(k)\right) ^H }{\left( \bar{\varvec{w}}_1^{(i)}(k)\right) ^H\varvec{Q}_y^{(i)}(k)\bar{\varvec{w}}_1^{(i)}(k)} \varvec{W}_{\perp }^{(i)}(k). \end{aligned}$$

Step3 For $i =2,\cdots , r$, update $\varvec{w}_i^{(1)}(k)$ as

$$\begin{aligned} \varvec{w}_i^{(1)}(k) = {\varvec{\perp }_i^{{\mathcal {A}}}}(k)\varvec{w}_1^{(i)}(k) = \varvec{W}_{\perp }^{(1)}(k)\cdots \varvec{W}_{\perp }^{(i-1)}(k)\varvec{w}_1^{(i)}(k). \end{aligned}$$