Markov Chain — Jemoka Knowledge Base

A Markov Chain is a chain of N states, with an N \times N transition matrix. at each step, we are in exactly one of those states the matrix P_{ij} tells us P(j|i), the probability of going to state j given you are at state i And therefore:

\begin{equation} \sum_{j=1}^{N} P_{ij} = 1 \end{equation}

Ergotic Markov Chain a markov chain is Ergotic if… you have a path from any one state to any other for any start state, after some time T_0, the probability of being in any state at any T > T_0 is non-zero Every Ergotic Markov Chain has a long-term visit rate: i.e. a steady state visitation count exists. We usually call it:

\begin{equation} \pi = \left(\pi_{i}, \dots, \pi_{n}\right) \end{equation}

Computing steady state Fact: let’s declare that \pi is the steady state to a transition matrix T; recall that the FROM states are the rows, which means that \pi has to be a row vector; \pi being a steady state makes:

\begin{equation} \pi T = \pi \end{equation}

This is a left e.v. with eigenvalue 1, which is the principle eigenvector of T as transition matricies always have eigenvector eigenvalue to 1.