Questions: Consistency of Estimators

Question 1

An estimator θ̂ₙ is known to be consistent for θ. A researcher applies it to a dataset with n=12 observations and obtains an estimate substantially far from the true θ. What should she conclude?

• A. The estimator is not actually consistent — a consistent estimator must produce close estimates even for small samples
• B. The model assumptions must be violated, since consistency would otherwise guarantee a good estimate
• C. Nothing contradictory — consistency only guarantees convergence in probability as n → ∞, and poor performance on a specific small sample is fully compatible with consistency
• D. The estimator has high bias, which by definition precludes consistency

Answer: (C)

Consistency is a purely asymptotic property: P(|θ̂ₙ − θ| > ε) → 0 as n → ∞. It makes no promise about any finite sample. For small n, a consistent estimator can be badly off — the probability of error is positive, just not converging to 1. This is one reason consistency alone is insufficient for a complete evaluation of an estimator; finite-sample properties like bias and mean squared error are separately important. A consistent estimator guarantees you'll do well *eventually*, not *now*.

Question 2

An estimator θ̂ₙ has bias equal to 1/n and variance equal to 1/n. Which statement is correct?

• A. The estimator is inconsistent — any nonzero bias disqualifies an estimator from being consistent
• B. The estimator is consistent only if it is also unbiased, which it is not at finite n
• C. The estimator is consistent — both bias and variance vanish as n → ∞, so by Chebyshev it converges in probability to θ
• D. Consistency cannot be determined from bias and variance alone

Answer: (C)

Consistency does not require unbiasedness at finite sample sizes. What matters is whether the estimator converges to the truth as n → ∞. Since bias = 1/n → 0 and variance = 1/n → 0, the MSE = bias² + variance = 1/n² + 1/n → 0. By Chebyshev's inequality (P(|θ̂ₙ − θ| > ε) ≤ MSE/ε²), this gives consistency. An estimator that is biased at every finite n but whose bias shrinks to zero is a common and valid class of consistent estimators.

Question 3

A consistent estimator should be unbiased for most finite sample sizes.

• T. True
• F. False

Answer: False

This is a common conflation. Unbiasedness (E[θ̂ₙ] = θ for all n) and consistency (convergence in probability as n → ∞) are different properties. A biased estimator can be consistent as long as the bias vanishes as n grows. Conversely, an unbiased estimator need not be consistent — if its variance doesn't go to zero, it will not converge in probability. The two properties are logically independent, though an unbiased estimator with vanishing variance is a sufficient condition for consistency.

Question 4

If an estimator is unbiased (E[θ̂ₙ] = θ for all n) and its variance converges to zero as n → ∞, then by Chebyshev's inequality it is consistent.

• T. True
• F. False

Answer: True

Chebyshev's inequality states P(|θ̂ₙ − θ| > ε) ≤ Var(θ̂ₙ)/ε². If the estimator is unbiased, then |θ̂ₙ − θ| has the same distribution as the centered version, and MSE = Var(θ̂ₙ). As Var(θ̂ₙ) → 0, the right-hand side goes to 0 for any fixed ε > 0, which is exactly the definition of convergence in probability. This gives a clean sufficient condition for consistency: zero bias plus vanishing variance suffices.

Question 5

What does consistency guarantee about an estimator, and what equally important properties does it NOT guarantee?

Model answer: Consistency guarantees that as sample size grows without bound, the probability of the estimator being far from the true parameter goes to zero — the estimator converges in probability to the truth. It does NOT guarantee good performance at any finite sample size, does not specify how quickly the estimator converges (the rate), does not require unbiasedness, and gives no information about the distribution of the estimator for constructing confidence intervals. Those properties — finite-sample behavior, convergence rate, and sampling distribution — are addressed by asymptotic normality and mean squared error analysis.

The conceptual point is that consistency is a floor, not a ceiling. It rules out estimators that stay wrong no matter how much data you have, but it doesn't distinguish among the many consistent estimators that differ dramatically in their efficiency (speed of convergence) and finite-sample properties. Asymptotic normality, which states √n(θ̂ₙ − θ) → N(0, σ²), is the next step: it quantifies how fast a consistent estimator converges and enables the construction of confidence intervals.