Questions: Markov Chains

Question 1

In a Markov chain, the probability of transitioning to set A at time n+1 depends on which of the following?

• A. The full history X₀, X₁, ..., Xₙ
• B. Only the current state Xₙ
• C. Only the initial state X₀
• D. The time-averaged state over all past steps

Answer: (B)

This is exactly the Markov property: P(Xₙ₊₁ ∈ A | X₀, ..., Xₙ) = P(Xₙ₊₁ ∈ A | Xₙ). The future distribution depends only on the present state, not the path taken to get there. This memoryless structure is what makes Markov chains analytically tractable.

Question 2

A time-homogeneous Markov chain is expected to have the same transition probabilities at most of the time step — this is part of the basic definition of a Markov chain.

• T. True
• F. False

Answer: False

Time-homogeneity is an additional assumption, not part of the bare definition. A Markov chain is defined solely by the Markov property (independence of the future from the past given the present). Non-homogeneous chains, where P(Xₙ₊₁ ∈ A | Xₙ = x) may vary with n, satisfy the Markov property but not time-homogeneity. Most classical theorems (stationary distributions, ergodicity) add time-homogeneity as a separate assumption.

Question 3

What information does the transition kernel P(x, A) encode, and why is it sufficient to fully specify a time-homogeneous Markov chain?

Model answer: P(x, A) gives the probability of moving to set A in one step from state x. Because the Markov property guarantees the future depends only on the current state, composing this one-step kernel with itself (the Chapman-Kolmogorov equations) determines the n-step distribution for any n.

Multi-step probabilities satisfy P^n(x, A) = ∫ P^(n-1)(y, A) P(x, dy). For countable state spaces this is matrix multiplication Pⁿ. The kernel contains everything needed because the Markov property collapses all history-dependence into the current state.