The Markov Chain Tree Theorem

Abstract

Let M be a finite first-order stationary Markov chain. We define an arborescence to be a set of edges in the directed graph for M having at most one edge out of every vertex, no cyles, and maximum cardinality. The weight of an arborescence is defined to be the product over each edge in the arborescence of the probability of the transition associated with the edge. We prove that if M starts in state i, its limiting average probability of being in state j is proportional to the sum of the weights of all arborescences having a path from i to j and no edge out of j. We present two proofs. The first is derived from simple graph theoretic identities. The second is derived from the closely-related Matrix Tree Theorem.