Memory-based random walk for multi-query local community detection

Abstract

Local community detection, which aims to find a target community containing a set of query nodes, has recently drawn intense research interest. The existing local community detection methods usually assume all query nodes are from the same community and only find a single target community. This is a strict requirement and does not allow much flexibility. In many real-world applications, however, we may not have any prior knowledge about the community memberships of the query nodes, and different query nodes may be from different communities. To address this limitation of the existing methods, we propose a novel memory-based random walk method, MRW, that can simultaneously identify multiple target local communities to which the query nodes belong. In MRW, each query node is associated with a random walker. Different from commonly used memoryless random walk models, MRW records the entire visiting history of each walker. The visiting histories of walkers can help unravel whether they are from the same community or not. Intuitively, walkers with similar visiting histories are more likely to be in the same community. Moreover, MRW allows walkers with similar visiting histories to reinforce each other so that they can better capture the community structure instead of being biased to the query nodes. We provide rigorous theoretical foundations for the proposed method and develop efficient algorithms to identify multiple target local communities simultaneously. Comprehensive experimental evaluations on a variety of real-world datasets demonstrate the effectiveness and efficiency of the proposed method.

Appendix

1.1 More theoretical results of the visiting history vector

In this section, we first show that the visiting history vector $\mathbf {v}^{(t)}$ (we omit the subscript “i” for simplicity) is a weighted sum of $\mathbf {e}^{(\tau )}(1\le \tau \le t)$. Then we provide a tighter error bound on $\Vert \mathbf {v}^{(t+1)}-\mathbf {v}^{(t)}\Vert _1$. These results are used in the speeding-up strategies introduced in Sect. 5.

1.1.1 Expansion of the visiting history vector

We expand the visiting history vector $\mathbf {v}^{(t)}$ into a linear combination of vectors $\mathbf {e}^{(\tau )}(0\le \tau \le t)$ and show that $\mathbf {v}^{(t)}$ is stochastic.

Theorem 4

Based on Eq. (3), for $K\ge 1$ and $t\ge 1$, we have

$$\begin{aligned} \mathbf {v}^{(t)} =\sum \limits _{\tau =0}^{t}\psi ^{(t)}_\tau (\beta )\mathbf {e}^{(\tau )} \end{aligned}$$

where

$$\begin{aligned} \psi ^{(t)}_{\tau }(\beta ) = {\left\{ \begin{array}{ll} 1-\frac{1}{K}\sum \limits _{j=1}^{K}\theta ^{(t)}_{t+1-j}(\beta )&{} \text {if } \tau =0\\ \frac{1}{K}[\theta ^{(t)}_{t+1-\tau }(\beta )-\theta ^{(t)}_{t+1-K-\tau }(\beta )] &{} \text {if } 1\le \tau \le t\\ \end{array}\right. } \end{aligned}$$

(8)

and

$$\begin{aligned} \theta ^{(t)}_{\tau }(\beta )= {\left\{ \begin{array}{ll} 1-\prod \limits _{j=1}^\tau (1-\beta ^{t-j}) &{} \text {if } 1\le \tau \le t\\ 0 &{} \text {otherwise}\\ \end{array}\right. } \end{aligned}$$

(9)

Please refer to Appendix 8.4 for the proof.

Next we show $\mathbf {v}^{(t)}$ is stochastic. Note that in the following analysis, we use $\theta ^{(t)}_{\tau }$ to represent $\theta ^{(t)}_{\tau }(\beta )$ for simplicity.

Lemma 2

Let $\varvec{\psi }^{(t)}$ be the column vector $[\psi ^{(t)}_{0},\dots ,\psi ^{(t)}_{t}]^\intercal$. For $t\ge 1$, $\varvec{\psi }^{(t)}$ is positive and $\parallel \varvec{\psi }^{(t)}\parallel _1=1$.

Proof

From Theorem 4, we have $0<\theta ^{(t)}_{\tau }\le 1$ for $1\le \tau \le t$ and $\theta ^{(t)}_{\tau }>\theta ^{(t)}_{\tau '}$ for $1\le \tau '<\tau \le t$. Thus, $\varvec{\psi }^{(t)}$ is positive. Furthermore, we have

$$\begin{aligned} \parallel \varvec{\psi }^{(t)}\parallel _1=1-\frac{1}{K}\sum \limits _{j=1}^{K}\theta ^{(t)}_{t+1-j}+\frac{1}{K}\sum \limits _{\tau =1}^{t}(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau }) \end{aligned}$$

If $1\le t\le K$,

$$\begin{aligned} \parallel \varvec{\psi }^{(t)}\parallel _1=1-\frac{1}{K}\sum \limits _{j=1}^{t}\theta ^{(t)}_{t+1-j}+\frac{1}{K}\sum \limits _{\tau =1}^{t}\theta ^{(t)}_{t+1-\tau }=1 \end{aligned}$$

If $t\ge K+1$,

$$\begin{aligned}&\parallel \varvec{\psi }^{(t)}\parallel _1\\&\quad = 1-\frac{1}{K}\sum \limits _{j=1}^{K}\theta ^{(t)}_{t+1-j}+\frac{1}{K}\sum \limits _{\tau =t-K+1}^{t}\theta ^{(t)}_{t+1-\tau }+\frac{1}{K}\sum \limits _{\tau =1}^{t-K}(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau })\\&\quad = 1+\frac{1}{K}\left( -\sum \limits _{\tau =t-K+1}^{t}\theta ^{(t)}_{\tau }+\sum \limits _{\tau =1}^{K}\theta ^{(t)}_{\tau }+\sum \limits _{\tau =K+1}^{t}\theta ^{(t)}_{\tau }-\sum \limits _{\tau =1}^{t-K}\theta ^{(t)}_{\tau }\right) \\&\quad = 1 \end{aligned}$$

Therefore, $\parallel \varvec{\psi }^{(t)}\parallel _1=1$. $\square$

The stochastic property of $\mathbf {v}^{(t)}$ can be directly derived from Lemma 2 and Eq. (7) which is summarized in the following theorem.

Theorem 5

For $t\ge 0$, $\mathbf {v}^{(t)}$ is nonnegative and $\parallel \mathbf {v}^{(t)}\parallel _1=1$.

1.1.2 A tighter error bound

Next we provide a tight error bound on $\parallel \mathbf {v}^{(t+1)}-\mathbf {v}^{(t)}\parallel _1$ to show its fast convergence. Based on Eqs. (8) and (9), when t is large, $\theta ^{(t)}_{\tau }(\beta )$ and $\psi ^{(t)}_{\tau }(\beta )$ will be small. For simplicity, in the following analysis, we use $\theta ^{(t)}_{\tau }$ and $\psi ^{(t)}_{\tau }$to represent $\theta ^{(t)}_{\tau }(\beta )$ and $\psi ^{(t)}_{\tau }(\beta )$, respectively. The following theorem first provides some properties of $\psi ^{(t)}_{\tau }$ regarding t and then gives the error bound on $\parallel \mathbf {v}^{(t+1)}-\mathbf {v}^{(t)}\parallel _1$.

Theorem 6

• (i) When $t>t_\beta =\log \epsilon /\log \beta$ for a $\epsilon >0$, we have $|\psi ^{(t+1)}_{0}-\psi ^{(t)}_{0}|<\epsilon$, $\psi ^{(t+1)}_{t+1}<\frac{\epsilon }{K}$ and $|\psi ^{(t+1)}_{\tau }-\psi ^{(t)}_{\tau }|<\frac{\epsilon }{K}$ for $1\le \tau \le t$.

• (ii) $\parallel \mathbf {v}^{(t+1)}-\mathbf {v}^{(t)}\parallel _1 \le 2\beta ^t(1-\frac{1}{K}\sum \nolimits _{\tau =1}^{K-1}\theta ^{(t)}_{\tau })$

To prove Theorem 6, we first show some properties of $\theta ^{(t)}_{\tau }$.

Lemma 3

For $t\ge \tau \ge 1$, $t>\tau _0\ge 1$, we have

• (i) $(1-\theta ^{(t)}_{\tau -1})(1-\beta ^{t-\tau }) = 1 - \theta ^{(t)}_{\tau }$

• (ii) $(1-\theta ^{(t)}_{\tau _0})(1-\theta ^{(t-\tau _0)}_{\tau }) = 1 - \theta ^{(t)}_{\tau _0+\tau }$

Lemma 3 can be directly derived from the definition of $\theta ^{(t)}_{\tau }$ in Eq. (9). From Lemma 3 we have the following additional properties.

Corollary 1

For $t\ge \tau \ge 1$, $t>\tau _0\ge 1$,

• (i) $\theta ^{(t)}_{1}=\beta ^{t-1}$; $\theta ^{(t)}_{t}=1$

• (ii) $(1-\theta ^{(t)}_{\tau -1})\beta ^{t-\tau } = \theta ^{(t)}_{\tau } - \theta ^{(t)}_{\tau -1}$

• (iii) $(1-\theta ^{(t)}_{\tau _0})\theta ^{(t-\tau _0)}_{\tau } = \theta ^{(t)}_{\tau _0+\tau } - \theta ^{(t)}_{\tau _0}$;

     $(1-\theta ^{(t-\tau _0)}_{\tau })\theta ^{(t)}_{\tau _0} = \theta ^{(t)}_{\tau _0+\tau } - \theta ^{(t-\tau _0)}_{\tau }$

Now we prove Theorem 6.

Proof

(of Theorem 6) Based on the properties of Corollary 1, we have $\theta ^{(t+1)}_{\tau +1}-\theta ^{(t)}_{\tau }=(1-\theta ^{(t)}_{\tau })\theta ^{(t+1)}_{1}$ for $t\ge \tau \ge 1$.

Now, we divide the proof into several cases based on the value of $\tau$.

• (a) When $\tau =0$, $\psi ^{(t)}_{0}=1-\frac{1}{K}\sum \limits _{j=1}^{K}\theta ^{(t)}_{t+1-j}$,

  $$\begin{aligned} \psi ^{(t+1)}_{0}-\psi ^{(t)}_{0}=\frac{-1}{K}\sum \limits _{j=1}^{K}(\theta ^{(t+1)}_{t+2-j}-\theta ^{(t)}_{t+1-j})=\frac{-1}{K}\sum \limits _{j=1}^{K}(1-\theta ^{(t)}_{t+1-j})\theta ^{(t+1)}_{1}<0 \end{aligned}$$

• (b) When $1\le \tau \le t-K$, $\psi ^{(t)}_{\tau }=\frac{1}{K}(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau })$,

  $$\begin{aligned} \psi ^{(t+1)}_{\tau }-\psi ^{(t)}_{\tau }&=\frac{1}{K}[(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t)}_{t+1-\tau })-(\theta ^{(t+1)}_{t+2-K-\tau }-\theta ^{(t)}_{t+1-K-\tau })]\\&=\frac{1}{K}[(1-\theta ^{(t)}_{t+1-\tau })-(1-\theta ^{(t)}_{t+1-K-\tau })]\theta ^{(t+1)}_{1}\\&=\frac{1}{K}(\theta ^{(t)}_{t+1-K-\tau }-\theta ^{(t)}_{t+1-\tau })\theta ^{(t+1)}_{1}\\&<0 \end{aligned}$$

• (c) When $\tau =t-K+1$,

  $$\begin{aligned} \psi ^{(t+1)}_{t-K+1}-\psi ^{(t)}_{t-K+1}&=\frac{1}{K}[(\theta ^{(t+1)}_{K+1}-\theta ^{(t+1)}_{1})-\theta ^{(t)}_{K}]\\&=\frac{1}{K}[(1-\theta ^{(t)}_{K})\theta ^{(t+1)}_{1}-\theta ^{(t+1)}_{1}]\\&=-\frac{1}{K}\theta ^{(t)}_{K}\theta ^{(t+1)}_{1}\\&<0 \end{aligned}$$

• (d) When $t-K+2\le \tau \le t$, $\psi ^{(t)}_{\tau }=\frac{1}{K}\theta ^{(t)}_{t+1-\tau }$,

  $$\begin{aligned} \psi ^{(t+1)}_{\tau }-\psi ^{(t)}_{\tau }&=\frac{1}{K}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t)}_{t+1-\tau })\\&=\frac{1}{K}(1-\theta ^{(t)}_{t+1-\tau })\theta ^{(t+1)}_{1}\\&>0 \end{aligned}$$

• (e) When $\tau =t+1$, $\psi ^{(t+1)}_{t+1}=\frac{1}{K}\theta ^{(t+1)}_{1}$.

For statement (i) in Theorem 6, when $t>t_\beta =\log \epsilon /\log \beta$, $\theta ^{(t+1)}_{1}=\beta ^t<\epsilon$, so $|\psi ^{(t+1)}_{0}-\psi ^{(t)}_{0}|<\epsilon$, $\psi ^{(t+1)}_{t+1}<\frac{\epsilon }{K}$ and $|\psi ^{(t+1)}_{\tau }-\psi ^{(t)}_{\tau }|<\frac{\epsilon }{K}$ for $1\le \tau \le t$.

For statement (ii), since $\mathbf {v}^{(t)} =\sum \nolimits _{\tau =0}^{t}\psi ^{(t)}_{\tau }\mathbf {e}^{(\tau )}$, we have

$$\begin{aligned}&\parallel \mathbf {v}^{(t+1)}-\mathbf {v}^{(t)}\parallel _1\\&\quad \le \psi ^{(t+1)}_{t+1}+\sum \limits _{\tau =0}^{t}|\psi ^{(t+1)}_{\tau }-\psi ^{(t)}_{\tau }|\\&\quad \le \frac{\theta ^{(t+1)}_{1}}{K}\left[ \sum \limits _{j=1}^{K}(1-\theta ^{(t)}_{t+1-j})\right. \\&\qquad \left. +\sum \limits _{\tau =1}^{t-K}(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau })+\theta ^{(t)}_{K}+\sum \limits _{\tau =t-K+2}^{t}(1-\theta ^{(t)}_{t+1-\tau })+1\right] \\&\quad =\frac{\theta ^{(t+1)}_{1}}{K}\left[ K-\sum \limits _{\tau =t-K+1}^{t}\theta ^{(t)}_{\tau }+\sum \limits _{\tau =K+1}^{t}\theta ^{(t)}_{\tau }-\sum \limits _{\tau =1}^{t-K}\theta ^{(t)}_{\tau }+\theta ^{(t)}_{K}-\sum \limits _{\tau =1}^{K-1}\theta ^{(t)}_{\tau }+K-1+1\right] \\&\quad =\frac{\theta ^{(t+1)}_{1}}{K}\left[ 2K-\sum \limits _{\tau =1}^{t}\theta ^{(t)}_{\tau }+\sum \limits _{\tau =K}^{t}\theta ^{(t)}_{\tau }+\sum \limits _{\tau =1}^{K-1}\theta ^{(t)}_{\tau }-2\sum \limits _{\tau =1}^{K-1}\theta ^{(t)}_{\tau }\right] \\&\quad =\frac{\theta ^{(t+1)}_{1}}{K}\left[ 2K-2\sum \limits _{\tau =1}^{K-1}\theta ^{(t)}_{\tau }\right] =2\beta ^t\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{K-1}\theta ^{(t)}_{\tau }\right) \end{aligned}$$

$\square$

1.2 The Proof of Theorem 2

In the following proof, we omit the subscript “i” for simplicity.

Proof

$$\begin{aligned}&\Vert \mathbf {x}^{(t)}-\mathbf {x}^{(t_\beta )}\Vert _1\\&\quad \le \Vert \mathbf {x}^{(t)}-\mathbf {x}^{(t-1)}\Vert _1+\cdots +\Vert \mathbf {x}^{(t_\beta +1)}-\mathbf {x}^{(t_\beta )}\Vert _1\\&\quad =\sum _{\tau =t_{\beta }+1}^{t}{\Delta }^{(\tau )} \end{aligned}$$

where ${\Delta }^{(\tau +1)}=\Vert \mathbf {x}^{(\tau +1)}-\mathbf {x}^{(\tau )}\Vert _1$.

From Lemma 1, we know that

$$\begin{aligned} {\Delta }^{(t+1)}\le {\left\{ \begin{array}{ll} \frac{2}{|\beta -\alpha |}\max \{\beta ^t,\alpha ^t\} &{} \text {if } \alpha \ne \beta \\ 2t\beta ^{t-1} &{} \text {if } \alpha =\beta \\ \end{array}\right. } \end{aligned}$$

If $\alpha \ne \beta$, let $c=\max \{\alpha ,\beta \}$, then

$$\begin{aligned}&\Vert \mathbf {x}^{(t)}-\mathbf {x}^{(t_\beta )}\Vert _1\\&\quad \le \sum _{\tau =t_{\beta }+1}^{t}{\Delta }^{(\tau )}=\frac{2}{|\beta -\alpha |}\sum _{\tau =t_{\beta }+1}^{t}c^{\tau -1}\\&\quad =\frac{2(c^{t_\beta }-c^t)}{|\beta -\alpha |(1-c)}\le \frac{2c^{t_\beta }}{|\beta -\alpha |(1-c)} \end{aligned}$$

Since $t_\beta =\log _\beta \epsilon$, we have $\alpha ^{t_\beta }=\epsilon ^{\log _\beta \alpha }$ and $\beta ^{t_\beta }=\epsilon$, then $c^{t_\beta }=O(\epsilon ^{\min \{1,\log _\beta \alpha \}})$

As a result, if $\alpha \ne \beta$, $\Vert \mathbf {x}^{(t)}-\mathbf {x}^{(t_\beta )}\Vert _1=O(\epsilon ^{\min \{1,\log _\beta \alpha \}})$.

If $\alpha =\beta$, we have

$$\begin{aligned} \Vert \mathbf {x}^{(t)}-\mathbf {x}^{(t_\beta )}\Vert _1&\le \sum _{\tau =t_{\beta }+1}^{t}{\Delta }^{(\tau )}=2\sum _{\tau =t_{\beta }}^{t-1}\tau \beta ^{\tau -1}\\&=\frac{2}{(1-\beta )^2}[\beta ^{t_\beta -1}(t_\beta (1-\beta )+\beta )-\beta ^{t-1}(t(1-\beta )+\beta )]\\&\le \frac{2}{(1-\beta )^2}(t_\beta \beta ^{t_\beta -1}+\beta ^{t_\beta })\\&=\frac{2}{(1-\beta )^2}\left( \frac{\epsilon \log \epsilon }{\beta \log \beta }+\epsilon \right) \end{aligned}$$

Therefore, $\Vert \mathbf {x}^{(t)}-\mathbf {x}^{(t_\beta )}\Vert _1=O(\epsilon |\log \epsilon |)$ if $\alpha =\beta$. $\square$

1.3 Convergence property of the reinforcement process

In this section, we show the convergence property of $\mathbf {x}_i^{(t)}$.

Theorem 7

$\lim \nolimits _{t\rightarrow \infty } \Vert \mathbf {x}_i^{(t+1)}-\mathbf {x}_i^{(t)}\Vert _1 = 0$ for $q_i\in Q$.

Proof

From $\mathbf {x}_i^{(t-1)}$ to $\mathbf {x}_i^{(t)}$, there are two steps: (1) form $\mathbf {x}_i^{(t-1)}$ to $\mathbb {x}_i^{(t-1)}$ based on Eq. (6), and (2) from $\mathbb {x}_i^{(t-1)}$ to $\mathbf {x}_i^{(t)}$ based on Eq. (2).

In the first step, we can rewrite Eq. (6) as $\mathbb {x}_i^{(t-1)}={\mathbb {Q}_i^{(t-1)}}^\intercal \mathbf {x}_i^{(t-1)}$, where

$$\begin{aligned} \mathbb {Q}_{i}^{(t)}= {\left\{ \begin{array}{ll} (1-\gamma )\mathbf {I}+\gamma \mathbf {1}{\mathbf {y}_{i}^{(t)}}^\intercal &{} \text {if } \sum \nolimits _{j=1}^S\mathbf {R}^{(t)}(j,i)=1\\ \mathbf {I} &{} \text {if } \sum \nolimits _{j=1}^S\mathbf {R}^{(t)}(j,i)=0\\ \end{array}\right. } \end{aligned}$$

(10)

and $\mathbf {y}_{i}^{(t)}=\sum \nolimits _{j=1}^S\mathbf {R}^{(t)}(j,i)\mathbf {x}_{j}^{(t)}$. We know that ${\mathbf {y}_{i}^{(t)}}^\intercal \mathbf {1}=1$.

In the second step, we can rewrite Eq. (2) as $\mathbf {x}_i^{(t)}={\mathbb {P}_i^{(t)}}^\intercal \mathbb {x}_i^{(t-1)}$, where $\mathbb {P}_i^{(t)}=\alpha \mathbf {P}+(1-\alpha )\mathbf {1}{\mathbf {v}_i^{(t-1)}}^\intercal$. According to Eq. (3), after a enough long time $\tau _1$, for $t>\tau _1$, $\mathbf {v}_i^{(t)}$ will converge, so will $\mathbb {P}_i^{(t)}$. Let $\mathbb {P}_i$ denote the converged $\mathbb {P}_i^{(t)}$.

As a result, $\mathbf {x}_i^{(t)}=\mathbb {P}_i^\intercal \mathbb {x}_i^{(t-1)}=[\mathbb {Q}_i^{(t-1)}\mathbb {P}_i]^\intercal \mathbf {x}_i^{(t-1)}$ and $\mathbb {Q}_i^{(t-1)}$ and $\mathbb {P}_i$ are stochastic, i.e., $\mathbb {Q}_i^{(t-1)}\mathbf {1}=\mathbb {P}_i\mathbf {1}=\mathbf {1}$.

Since the reinforcement process is a dynamic process between time points 0 and t, we can construct a graph to record the intermediate steps as follows. If at certain time point $\tau$ ($0\le \tau \le t$), the score vectors of $q_i$ and $q_j$ have positive similarity (bigger than a very small threshold, e.g., 0.01), then there is an edge between them. In this graph, if a node $q_i$ is isolated, that means the score vector of $q_i$ has no positive similarity (bigger than a very small threshold, e.g., 0.01) with the score vector of any other query node at all time. Its convergence is guaranteed according to Theorem 1.

Let $Q'$ represent the node set of any other connected component containing at least 2 nodes. For a query node $q_i\in Q'$, let $T=\{\tau |\sum \nolimits _{q_j\in Q'}\mathbf {R}^{(\tau )}(j,i)=0\}$. Then for $\tau \in T$, $\mathbb {Q}_i^{(\tau )}=\mathbf {I}$. Note that $T\ne \emptyset$, since we have $\mathbb {Q}_i^{(0)}=\mathbf {I}$. Next we extend $\mathbf {x}_i^{(t)}$.

$$\begin{aligned} \mathbf {x}_i^{(t)}&=[\mathbb {Q}_i^{(t-1)}\mathbb {P}_i]^\intercal \mathbf {x}_i^{(t-1)}=[\mathbb {Q}_i^{(t-2)}\mathbb {P}_i\mathbb {Q}_i^{(t-1)}\mathbb {P}_i]^\intercal \mathbf {x}_i^{(t-2)}\\&=[\mathbb {Q}_i^{(t-2)}[(1-\gamma )\mathbb {P}_i+\gamma \mathbf {1}{\mathbf {y}_i^{(t-1)}}^\intercal ]\mathbb {P}_i]^\intercal \mathbf {x}_i^{(t-2)}\\&=[(1-\gamma )\mathbb {Q}_i^{(t-2)}\mathbb {P}_i^2+\gamma \mathbf {1}{\mathbf {y}_i^{(t-1)}}^\intercal \mathbb {P}_i]^\intercal \mathbf {x}_i^{(t-2)}\\&=[(1-\gamma )^{t-1-|T|}\mathbb {Q}_i^{(0)}\mathbb {P}_i^t+\gamma \sum \limits _{k=1,k\notin T}^{t-1}(1-\gamma )^{k-1}\mathbf {1}{\mathbf {y}_i^{(t-k)}}^\intercal \mathbb {P}_i^k]^\intercal \mathbf {x}_i^{(0)} \end{aligned}$$

Based on the Perron–Frobenius theorem, after a enough long time $\tau _2$, for $t>\tau _2$, $\mathbb {P}_i^t=\mathbf {1x}_i^\intercal$, i.e., the row vectors of $\mathbb {P}_i^t$ will be an identical vector $\mathbf {x}_i^\intercal$ [13]. We set $\tau _0 = \max \{\tau _1,\tau _2\}$. Then when $t>\tau _0$, we have

$$\begin{aligned} \mathbf {x}_i^{(t)} =&\,(1-\gamma )^{t-1-|T|}\mathbf {x}_i+\gamma \sum \limits _{k=\tau _0+1,k\notin T}^{t-1}(1-\gamma )^{k-1}\mathbf {x}_i\\&+\gamma \sum \limits _{k=1,k\notin T}^{\tau _0}(1-\gamma )^{k-1}[\mathbb {P}_i^k]^\intercal {\mathbf {y}_i^{(t-k)}}\\ \end{aligned}$$

Next we check the difference between $\mathbf {x}_i^{(t+1)}$ and $\mathbf {x}_i^{(t)}$.

$$\begin{aligned} \Delta _i^{(t+1)}&=\Vert \mathbf {x}_i^{(t+1)}-\mathbf {x}_i^{(t)}\Vert _1\\&\le -\gamma (1-\gamma )^{t-1-|T|}+\gamma (1-\gamma )^{t-1}+\gamma \sum \limits _{k=1,k\notin T}^{\tau _0}(1-\gamma )^{k-1}\Vert \mathbf {y}_i^{(t+1-k)}-\mathbf {y}_i^{(t-k)}\Vert _1\\&\le \gamma \sum \limits _{k=1,k\notin T}^{\tau _0}(1-\gamma )^{k-1}\Vert \mathbf {y}_i^{(t+1-k)}-\mathbf {y}_i^{(t-k)}\Vert _1 \end{aligned}$$

Since after $t_c$ steps, $\mathbf {R}^{(t)}$ does not change, we have

$$\begin{aligned}&\Vert \mathbf {y}_i^{(t+1)}-\mathbf {y}_i^{(t)}\Vert _1\\&\quad =\sum \limits _{q_j\in Q'}\Vert \mathbf {R}^{(t+1)}(j,i)\mathbf {x}_j^{(t+1)}-\mathbf {R}^{(t)}(j,i)\mathbf {x}_{j}^{(t)}\Vert _1\\&\quad =\sum \limits _{q_j\in Q'}\mathbf {R}^{(t_c)}(j,i)\Vert \mathbf {x}_j^{(t+1)}-\mathbf {x}_{j}^{(t)}\Vert _1\\&\quad \le \max _{q_j\in {Q'}}\Delta _j^{(t+1)} \end{aligned}$$

Let $\Delta ^{(t+1)}=\max _{q_j\in {Q'}}\Delta _j^{(t+1)}$, we have

$$\begin{aligned} \Delta ^{(t+1)}\le \gamma \sum \limits _{k=1,k\notin T}^{\tau _0}(1-\gamma )^{k-1}\Delta ^{(t+1-k)}<\gamma \sum \limits _{k=1}^{\tau _0}(1-\gamma )^{k-1}\Delta ^{(t+1-k)} \end{aligned}$$

Let $\Delta '^{(t+1)}=\gamma \sum \nolimits _{k=1}^{\tau _0}(1-\gamma )^{k-1}\Delta ^{(t+1-k)}$, we have $\Delta ^{(t+1)}<\Delta '^{(t+1)}$. Moreover,

$$\begin{aligned}&\Delta '^{(t+1)}\\&\quad =\gamma \sum \nolimits _{k=1}^{\tau _0}(1-\gamma )^{k-1}\Delta ^{(t+1-k)}\\&\quad =\gamma \Delta ^{(t)}+(1-\gamma )\left[ \gamma \sum \limits _{k=2}^{\tau _0+1}(1-\gamma )^{k-2}\Delta ^{(t+1-k)}\right] -\gamma (1-\gamma )^{\tau _0}\Delta ^{(t-\tau _0)}\\&\quad =\gamma \Delta ^{(t)}+(1-\gamma )\Delta '^{(t)}-\gamma (1-\gamma )^{\tau _0}\Delta ^{(t-\tau _0)}\\&\quad<\gamma \Delta '^{(t)}+(1-\gamma )\Delta '^{(t)}\\&\quad <\Delta '^{(t)} \end{aligned}$$

Thus, for any $q_i\in Q'$, $\lim \nolimits _{t\rightarrow \infty }\Delta _i^{(t)}=\lim \nolimits _{t_\rightarrow \infty }\Delta ^{(t)}=\lim \nolimits _{t_\rightarrow \infty }\Delta '^{(t)}=0$. $\square$

1.4 The Proof of Theorem 4

Proof

We use induction to prove

$$\begin{aligned} \mathbf {v}^{(t)} = \left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{K}\theta ^{(t)}_{t+1-\tau }\right) \mathbf {e}^{(0)}+\frac{1}{K}\sum \limits _{\tau =1}^{t}(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau })\mathbf {e}^{(\tau )} \end{aligned}$$

(11)

When $t=1$, based on Eq. (3) and $\mathbf {v}^{(0)}=\mathbf {e}^{(0)}$, we have

$$\begin{aligned} \mathbf {v}^{(1)}= & {} (1-\beta ^{1-1})\mathbf {v}^{(0)}+\beta ^{1-1}\frac{1}{K}(\mathbf {e}^{(1)}+\mathbf {e}^{(0)}+\cdots +\mathbf {e}^{(0)})\\= & {} \frac{K-1}{K}\mathbf {e}^{(0)}+\frac{1}{K}\mathbf {e}^{(1)} \end{aligned}$$

On the other hand, based on Eq. (11), we have

$$\begin{aligned} \mathbf {v}^{(1)} = \left( 1-\frac{\theta ^{(1)}_{2-1}}{K}\right) \mathbf {e}^{(0)}+\frac{\theta ^{(1)}_{2-1}}{K}\mathbf {e}^{(1)}=\frac{K-1}{K}\mathbf {e}^{(0)}+\frac{1}{K}\mathbf {e}^{(1)} \end{aligned}$$

Suppose that for $t\ge 1$, Eq. (11) holds. Now we check $\mathbf {v}^{(t+1)}$. When $1\le t\le K-2$,

$$\begin{aligned} \mathbf {v}^{(t+1)}=(1-\beta ^{t})\mathbf {v}^{(t)}+\frac{\beta ^{t}}{K}\left( \sum \limits _{\tau =1}^{t+1}\mathbf {e}^{(\tau )}+(K-t-1)\mathbf {e}^{(0)}\right) \end{aligned}$$

Since $t\le K-2$, then for $1\le \tau \le t$, $\theta ^{(t)}_{t+1-K-\tau }=0$. Next, based on Eq. (11),

$$\begin{aligned} \mathbf {v}^{(t)}= \left( 1-\frac{1}{K}\sum \limits _{i=1}^{t}\theta ^{(t)}_{t+1-\tau }\right) \mathbf {e}^{(0)}+\frac{1}{K}\sum \limits _{\tau =1}^{t}\theta ^{(t)}_{t+1-\tau }\mathbf {e}^{(\tau )} \end{aligned}$$

According to Corollaries 1 (i) and (iii), we have $\beta ^t=\theta ^{(t+1)}_{1}$ and $(1-\theta ^{(t+1)}_{1})\theta ^{(t+1-1)}_{t+1-\tau }=\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{1}$ for $1\le \tau \le t$. Then we substitute $\mathbf {v}^{(t)}$ into $\mathbf {v}^{(t+1)}$.

$$\begin{aligned} \mathbf {v}^{(t+1)}&=(1-\theta ^{(t+1)}_{1})\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{t}\theta ^{(t)}_{t+1-\tau }\right) \mathbf {e}^{(0)}+\frac{1-\theta ^{(t+1)}_{1}}{K}\sum \limits _{\tau =1}^{t}\theta ^{(t)}_{t+1-\tau }\mathbf {e}^{(\tau )}\\&\quad +\theta ^{(t+1)}_{1}\frac{K-t-1}{K}\mathbf {e}^{(0)}+\frac{1}{K}\theta ^{(t+1)}_{1}\sum \limits _{\tau =1}^{t+1}\mathbf {e}^{(\tau )}\\&=\left[ 1-\theta ^{(t+1)}_{1}\left( 1-\frac{K-t-1}{K}\right) -\frac{1}{K}\sum \limits _{\tau =1}^{t}(1-\theta ^{(t+1)}_{1})\theta ^{(t)}_{t+1-\tau }\right] \mathbf {e}^{(0)}\\&\quad +\frac{1}{K}\sum \limits _{\tau =1}^{t}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{1})\mathbf {e}^{(\tau )}+\frac{\theta ^{(t+1)}_{1}}{K}\sum \limits _{\tau =1}^{t}\mathbf {e}^{(\tau )}+\frac{\theta ^{(t+1)}_{1}}{K}\mathbf {e}^{(t+1)}\\&=\left[ 1-\theta ^{(t+1)}_{1}\left( 1-\frac{K-t-1}{K}-\frac{t}{K}\right) -\frac{1}{K}\sum \limits _{\tau =1}^{t}\theta ^{(t+1)}_{t+2-\tau }\right] \mathbf {e}^{(0)}\\&\quad +\frac{1}{K}\sum \limits _{\tau =1}^{t}\theta ^{(t+1)}_{t+2-\tau }\mathbf {e}^{(\tau )}+\frac{1}{K}\theta ^{(t+1)}_{1}\mathbf {e}^{(t+1)}\\&=\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{t+1}\theta ^{(t+1)}_{t+2-\tau }\right) \mathbf {e}^{(0)}+\frac{1}{K}\sum \limits _{\tau =1}^{t+1}\theta ^{(t+1)}_{t+2-\tau }\mathbf {e}^{(\tau )}\\ \end{aligned}$$

This means when $1\le t\le K-1$, Eq. (11) holds. In the following inductive steps, we check $\mathbf {v}^{(t+1)}$ for $t\ge K-1$.

Based on Eq. (3), we have

$$\begin{aligned} \mathbf {v}^{(t+1)} =&(1-\beta ^{t})\mathbf {v}^{(t)}+\beta ^{t}\frac{1}{K}(\mathbf {e}^{(t+1)}+\mathbf {e}^{(t)}+\cdots + \mathbf {e}^{(t+2-K)})\\ =&(1-\theta ^{(t+1)}_{1})\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{K}\theta ^{(t)}_{t+1-\tau }\right) \mathbf {e}^{(0)}\\&+\frac{(1-\theta ^{(t+1)}_{1})}{K}\sum \limits _{\tau =1}^{t}(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau })\mathbf {e}^{(\tau )}\\&+\frac{1}{K}\theta ^{(t+1)}_{1}\mathbf {e}^{(t+1)}+\frac{1}{K}\theta ^{(t+1)}_{1}\sum \limits _{\tau =t-K+2}^{t}\mathbf {e}^{(\tau )} \end{aligned}$$

If $t-K+1\le \tau \le t$, then $\theta ^{(t)}_{t+1-K-\tau }=0$. Now we divide the second term into two parts: $1\sim (t-K)$ and $(t-K+1)\sim t$.

$$\begin{aligned} \mathbf {v}^{(t+1)}&=\left[ 1-\theta ^{(t+1)}_{1}-\frac{1}{K}\sum \limits _{\tau =1}^{K}(1-\theta ^{(t+1)}_{1})\theta ^{(t)}_{t+1-\tau }\right] \mathbf {e}^{(0)}\\&\quad +\frac{1}{K}\sum \limits _{\tau =1}^{t-K}(1-\theta ^{(t+1)}_{1})(\theta ^{(t)}_{t+1-\tau }-\theta ^{(t)}_{t+1-K-\tau })\mathbf {e}^{(\tau )}\\&\quad +\frac{1}{K}\sum \limits _{\tau =t-K+1}^{t}(1-\theta ^{(t+1)}_{1})\theta ^{(t)}_{t+1-\tau }\mathbf {e}^{(\tau )}+\frac{1}{K}\theta ^{(t+1)}_{1}\mathbf {e}^{(t+1)}+\frac{1}{K}\theta ^{(t+1)}_{1}\sum \limits _{\tau =t-K+2}^{t}\mathbf {e}^{(\tau )}\\&=\left[ 1-\theta ^{(t+1)}_{1}-\frac{1}{K}\sum \limits _{\tau =1}^{K}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{1})\right] \mathbf {e}^{(0)}\\&\quad +\frac{1}{K}\sum \limits _{\tau =1}^{t-K}[(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{1})-(\theta ^{(t+1)}_{t+2-K-\tau }-\theta ^{(t+1)}_{1})]\mathbf {e}^{(\tau )}\\&\quad +\frac{1}{K}\theta ^{(t+1)}_{1}\mathbf {e}^{(t+1)}+\frac{1}{K}\sum \limits _{\tau =t-K+1}^{t}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{1})\mathbf {e}^{(\tau )}+\frac{1}{K}\theta ^{(t+1)}_{1}\sum \limits _{\tau =t-K+2}^{t}\mathbf {e}^{(\tau )}\\&=\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{K}\theta ^{(t+1)}_{t+2-\tau }\right) \mathbf {e}^{(0)}+\frac{1}{K}\sum \limits _{\tau =1}^{t-K}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{t+2-K-\tau })\mathbf {e}^{(\tau )}+\frac{1}{K}\theta ^{(t+1)}_{1}\mathbf {e}^{(t+1)}\\&\quad +\frac{1}{K}\sum \limits _{\tau =t-K+2}^{t}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{1}+\theta ^{(t+1)}_{1})\mathbf {e}^{(\tau )}+\frac{1}{K}(\theta ^{(t+1)}_{K+1}-\theta ^{(t+1)}_{1})\mathbf {e}^{(t-K+1)}\\&=\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{K}\theta ^{(t+1)}_{t+2-\tau }\right) \mathbf {e}^{(0)}+\frac{1}{K}\sum \limits _{\tau =1}^{t-K+1}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{t+2-K-\tau })\mathbf {e}^{(\tau )}\\&\quad +\frac{1}{K}\sum \limits _{\tau =t-K+2}^{t+1}\theta ^{(t+1)}_{t+2-\tau }\mathbf {e}^{(\tau )} \end{aligned}$$

Since when $t-K+2\le \tau \le t+1$, $\theta ^{(t+1)}_{t+2-K-\tau }=0$, we can combine the last two items to get

$$\begin{aligned} \mathbf {v}^{(t+1)} =\left( 1-\frac{1}{K}\sum \limits _{\tau =1}^{K}\theta ^{(t+1)}_{t+2-\tau }\right) \mathbf {e}^{(0)}+\frac{1}{K}\sum \limits _{\tau =1}^{t+1}(\theta ^{(t+1)}_{t+2-\tau }-\theta ^{(t+1)}_{t+2-K-\tau })\mathbf {e}^{(\tau )} \end{aligned}$$

Therefore, when $t\ge 1$, Eq. (11) is always valid. $\square$