For a finite, irreducible, aperiodic Markov chain with stationary distribution π, you run the chain for a very long time starting from state i. As the number of steps n → ∞, the probability of being in state j approaches:
AP_{ij}, the one-step transition probability from i to j
Bπ_j, the stationary probability of state j, regardless of the starting state i
C1/k where k is the total number of states, since the chain visits all states equally
DThe probability depends on which state i you started in — there is no universal limit
The fundamental convergence theorem for Markov chains: for a finite irreducible aperiodic chain, P^n_{ij} → π_j as n → ∞, for every starting state i. The chain 'forgets' its initial state and converges to the same limiting distribution π regardless of where it started. This is not obvious — it requires both irreducibility (so every state is reachable) and aperiodicity (so there is no cyclic trapping). The stationary distribution is not just a fixed point but the universal attractor of the dynamics.
Question 2 Multiple Choice
A Markov chain satisfies detailed balance: π_i P_{ij} = π_j P_{ji} for all states i and j. What can you conclude?
AThe chain is irreducible and will converge to π from any starting distribution
Bπ is a stationary distribution for the chain, but detailed balance alone does not guarantee irreducibility or convergence
CThe chain must be reversible and have uniform stationary distribution
DDetailed balance is equivalent to the chain being aperiodic
Detailed balance implies stationarity: summing π_i P_{ij} = π_j P_{ji} over all i gives (πP)_j = π_j, so π is stationary. However, detailed balance says nothing about irreducibility (whether all states communicate) or aperiodicity (whether the chain lacks periodic cycles). A chain could satisfy detailed balance while being decomposable into separate communicating classes, each with its own stationary distribution. Detailed balance is a sufficient condition for finding stationary distributions, not a guarantee of convergence to a unique one.
Question 3 True / False
If you start a Markov chain in its stationary distribution π, the distribution of the chain's state at every future time step remains π.
TTrue
FFalse
Answer: True
True — this is precisely what 'stationary' means. A distribution π is stationary if πP = π: applying the transition matrix once leaves π unchanged. Therefore if the chain starts with distribution π₀ = π, then after one step the distribution is π₀P = πP = π, and by induction it remains π at every step. The chain in stationarity is in a kind of probabilistic equilibrium: probabilities of being in each state do not change over time, even though individual realizations of the chain continue to transition between states.
Question 4 True / False
A finite irreducible Markov chain can have two distinct stationary distributions if its transition probabilities are chosen carefully.
TTrue
FFalse
Answer: False
False. For a finite irreducible chain, the stationary distribution is unique. Irreducibility means all states communicate (every state is reachable from every other), which ensures the chain cannot be decomposed into independent subchains each with its own stationary distribution. The stationary distribution for an irreducible chain satisfies π_j = 1/⟨T_j⟩ (the reciprocal of the mean return time to state j), which uniquely determines π. Adding aperiodicity ensures convergence to this unique stationary distribution, but uniqueness holds for irreducible chains regardless of periodicity.
Question 5 Short Answer
Explain why the existence of a stationary distribution makes Markov Chain Monte Carlo (MCMC) a practical algorithm for sampling from complex probability distributions.
Think about your answer, then reveal below.
Model answer: MCMC works by constructing a Markov chain whose stationary distribution is exactly the target distribution you want to sample from (e.g., a Bayesian posterior). By the convergence theorem, the chain will eventually produce samples that look like draws from that target distribution, regardless of where it started. Even if direct sampling from the target is computationally intractable, you can often design a transition kernel satisfying detailed balance with respect to the target — ensuring stationarity — and then sample by simulating the chain past its mixing time.
The key insight is the reversal of the usual relationship: instead of starting with a chain and finding its stationary distribution, MCMC starts with a desired distribution and constructs a chain with that distribution as its stationary distribution. Detailed balance provides the design criterion: any transition kernel satisfying π_i P_{ij} = π_j P_{ji} has π as its stationary distribution. Algorithms like Metropolis-Hastings do exactly this, accepting or rejecting proposed moves in a way that guarantees detailed balance is satisfied. The only requirement for validity is that the chain is irreducible and aperiodic, so it actually converges.