Aggregation and Truncation of Reversible Markov Chains Modulo State Renaming

Abstract

The theory of time-reversibility has been widely used to derive the expressions of the invariant measures and, consequently, of the equilibrium distributions for a large class of Markov chains which found applications in optimisation problems, computer science, physics, and bioinformatics. One of the key-properties of reversible models is that the truncation of a reversible Markov chain is still reversible. In this work we consider a more general notion of reversibility, i.e., the reversibility modulo state renaming, called $\rho$-reversibility, and show that some of the properties of reversible chains cannot be straightforwardly extended to $\rho$-reversible ones. Among these properties, we show that in general the truncation of the state space of a $\rho$-reversible chain is not $\rho$-reversible. Hence, we derive further conditions that allow the formulation of the well-known properties of reversible chains for $\rho$-reversible Markov chains. Finally, we study the properties of the state aggregation in $\rho$-reversible chains and prove that there always exists a state aggregation that associates a $\rho$-reversible process with a reversible one.

A Proofs of the Results

1.1 Proof of Theorem 1

Proof

By Proposition 1 and Definition 1, to prove that $\widetilde{X}(t)$ is $\widetilde{\rho }$-reversible it is sufficient to show that for all $S_i,S_j\in \mathcal S/{\sim }$ with $i\not =j$,

$$ \widetilde{\pi }(S_i) \widetilde{q}(S_i,S_j) =\widetilde{\pi }({\widetilde{\rho }(S_j)}) \widetilde{q}({\widetilde{\rho }(S_j)},\widetilde{\rho }(S_i)). $$

By Eq. (3) and Proposition 4, this is equivalent to:

which can be written as:

$$\begin{aligned} \sum _{s\in S_i}\sum _{s'\in S_j}\pi (s)q(s,s')= \sum _{s'\in \widetilde{\rho }(S_j)}\sum _{s\in \widetilde{\rho }(S_i)}\pi (s')q(s',s). \end{aligned}$$

(7)

We now proceed by considering four cases.

• 1. Assume that $S_i=\{s\}$ and $S_j=\{s'\}$, then Eq. (7) becomes:

  $$ \pi (s)q(s,s')=\pi ({\rho (s')})q({\rho (s')},\rho (s)), $$

  where we have used the definition of $\widetilde{\rho }$ for singletons. This is true since by hypothesis X(t) is $\rho$-reversible and hence satisfies the $\rho$-detailed balance equation.

• 2. Assume $S_i=\{s\}$ and $|S_j|>1$, and recall that $\widetilde{\rho }(S_j)=S_j$ by definition. Then Eq. (7) can be rewritten as:

  $$ \sum _{s' \in S_j}\pi (s)q(s,s')=\sum _{s' \in S_j} \pi (s')q(s',\rho (s)). $$

  Since $\sim$ respects $\rho$ we have that $\rho$ restricted to the elements of $S_j$ is still a bijection and hence we can write:

  $$ \sum _{s' \in S_j}\pi (s)q(s,s')=\sum _{s' \in S_j} \pi ({\rho (s')})q({\rho (s')},\rho (s)), $$

  which is true by the hypothesis of $\rho$-reversibility of X(t).

• 3. Assume $|S_i|>1$ and hence $\widetilde{\rho }(S_i)=S_i$ and $S_j=\{s'\}$, then Eq. (7) can be written as:

  $$ \sum _{s \in S_i}\pi (s)q(s,s')=\sum _{s \in S_i}\pi ({\rho (s')})q({\rho (s')},s). $$

  Since $\rho$ restricted to the elements of $S_i$ is a bijection, then we have:

  $$ \sum _{s \in S_i}\pi (s)q(s,s')=\sum _{s \in S_i}\pi ({\rho (s')})q({\rho (s')},\rho (s)), $$

  which is an identity.

• 4. Assume $|S_i|>1$ and $|S_j|>1$, and hence $\widetilde{\rho }(S_i)=S_i$ and $\widetilde{\rho }(S_j)=S_j$. Then we can rewrite Eq. (7) as:

  $$ \sum _{s \in S_i} \sum _{s' \in S_j}\pi (s)q(s,s') = \sum _{s \in S_i}\sum _{s' \in S_j} \pi (s')q(s',s). $$

  Since $\rho$ restricted to $S_i$ and to $S_j$ is still a bijection because $\sim$ respects $\rho$, we can rewrite the previous equation as:

  $$ \sum _{s \in S_i} \sum _{s' \in S_j}\pi (s)q(s,s') = \sum _{s \in S_i}\sum _{s' \in S_j} \pi ({\rho (s')})q({\rho (s')},\rho (s)). $$

  which is true by hypothesis.    $\square$

1.2 Proof or Proposition 5

Proof

By the general aggregation Eq. (3), for any $S_i, S_j \in \mathcal S/{\sim }$,

$$\begin{aligned} \widetilde{q}(S_i,S_j)=\frac{\sum _{s'\in S_i}\pi (s')\sum _{s\in S_j}q(s',s)}{\sum _{s'\in S_i}\pi (s')}, \end{aligned}$$

(8)

Since X(t) is $\rho$-reversible and each $S_i\in S/{\sim }$ is an orbit for $\rho$, it holds that $\pi (s)=\pi (s')$ for all $s,s'\in S_i$. Let us denote by $\pi (S_i)$ the equilibrium probability of each s belonging to the orbit $S_i$. Hence, $\sum _{s'\in S_i}\pi (s')=|S_i|\pi (S_i)$ and Eq. (8) can be written

$$\begin{aligned} \widetilde{q}(S_i,S_j)=\pi (S_i)\frac{\sum _{s'\in S_i}\sum _{s\in S_j}q(s',s)}{|S_i|\pi (S_i)} \end{aligned}$$

(9)

proving the statement.    $\square$

1.3 Proof of Lemma 1

Proof

To prove the lemma we use Proposition 1. In fact, let us consider two states $s,u \in \mathcal A$, then the corresponding $\rho$-detailed balance equation is $B \pi (s) q(s,u)=B\pi ({\rho (u)})q({\rho (u)},\rho (s))$ since we have by assumption that the partition respects $\rho$ and hence also $\rho (t),\rho (s) \in \mathcal A$. This equation is satisfied because X(t) is $\rho$-reversible. If $s,u \in \mathcal S\smallsetminus \mathcal A$ the corresponding detailed balance equation is $Bc \pi (s) q(s,u)=Bc\pi ({\rho (u)})q({\rho (u)},\rho (s))$ that is also satisfied for the same reasons. Let us consider $s \in \mathcal A$ and $u \in \mathcal S \smallsetminus \mathcal A$, then we have that the transition rates are modified and hence $B \pi (s) \left( c q(s,u)\right) =Bc\pi ({\rho (u)})q({\rho (u)},\rho (s))$ which is an identity since $\sim$ respects $\rho$. Finally, we have to consider the case of $s \in \mathcal S\smallsetminus \mathcal A$ and $u \in \mathcal A$. The corresponding detailed balance equation is $B c \pi (s) q(s,u)=B\pi ({\rho (u)})\left( cq({\rho (u)},\rho (s))\right)$ which is satisfied by hypothesis. The fact that the residence times in the states belonging to the same orbits of $\rho$ in $X'(t)$ are identically distributed is an assumption of the lemma.   $\square$