In a CTMC, the holding time in state i is Exponential with rate qᵢ = Σ_{j≠i} qᵢⱼ. Why must the holding time be exponential for the Markov property to hold in continuous time?
ABecause the exponential distribution is the only continuous distribution with the memoryless property — P(T > t+s | T > t) = P(T > s)
BBecause the exponential distribution has the smallest variance among positive distributions, ensuring stability
CBecause the generator matrix requires positive off-diagonal entries, which forces exponential holding times
DAny distribution would work; the exponential is chosen for mathematical convenience
The Markov property requires that the future depends only on the current state, not on how long the process has been in that state. If the holding time were non-exponential (say, uniformly distributed), then knowing the process has been in state i for time s would give information about the remaining holding time — violating the Markov property. The memoryless property of the exponential distribution ensures that 'time already spent' is irrelevant, making the current state a sufficient summary. This is the fundamental reason CTMCs must have exponential holding times.
Question 2 Multiple Choice
The generator matrix Q of a CTMC has the property that each row sums to zero: Σⱼ qᵢⱼ = 0 for all i. This is analogous to what property of the transition matrix P in discrete-time Markov chains?
AEach row of P sums to 1 (rows are probability distributions) — the zero row sums of Q reflect the fact that Q generates the rate of change, not the distribution itself
BP is symmetric — the zero row sums enforce reversibility
CP has eigenvalue 1 — the zero row sums ensure Q has eigenvalue 0
DBoth A and C are correct
The transition matrix P = e^{Qt} must have rows summing to 1 (it's a stochastic matrix). This requires Q to have rows summing to 0: the diagonal entry qᵢᵢ = -Σ_{j≠i} qᵢⱼ is the negative total exit rate. Equivalently, Q1 = 0 (the vector of ones is in the null space), which means 0 is an eigenvalue of Q — analogous to 1 being an eigenvalue of P. The relationship P(t) = e^{Qt} connects the discrete snapshot (transition probabilities over time t) to the continuous dynamics (instantaneous rates).
Question 3 Short Answer
A two-state CTMC has states {0,1} with transition rates q₀₁ = α and q₁₀ = β. Find the stationary distribution.
Think about your answer, then reveal below.
Model answer: The stationary distribution π = (π₀, π₁) satisfies πQ = 0 and π₀ + π₁ = 1. The generator Q has q₀₁ = α, q₁₀ = β, q₀₀ = -α, q₁₁ = -β. The equation π₀(-α) + π₁β = 0 gives π₀α = π₁β, so π₀/π₁ = β/α. With normalization: π₀ = β/(α+β) and π₁ = α/(α+β). The process spends time in each state proportional to the reciprocal of its exit rate — the state with slower exit rate (lower transition rate out) has higher stationary probability.
This is the continuous-time analogue of the discrete-time two-state chain. The stationary distribution depends on the ratio of transition rates, not their absolute magnitudes. Doubling both α and β speeds up the process but doesn't change the long-run fraction of time spent in each state. The absolute rates control the mixing time (how fast the process approaches stationarity), while the ratios control the stationary distribution.