A Combinatorial Aggregation Algorithm for Stationary Distribution of a Large Markov Chain

Abstract

A new aggregation algorithm for computing the stationary distribution of a large Markov chain is proposed. This algorithm is attractive when the state space of Markov chain is large enough so that the direct and iterative methods are inefficient. It is based on grouping the states of a Markov chain in such a way that the probability of changing the state inside the group is of greater order of magnitude than interactions between groups. The correctness of the combinatorial aggregation is justified by the method of forest expansions developed recently in [5,6]. In contrast to existing methods our approach is based on combinatorial and graph-theoretic framework and can be seen as an algorithmization of famous Markov Chain Tree Theorem. The general method is illustrated by an example of computing the stationary distribution. We establish also some preliminary results on the complexity of our algorithm. Numerical experiments on several benchmark examples show the potential applicability of the algorithm in real life problems.

This work was partially supported by the KBN grant 8 T11C 039 15