Questions: Adjusted R-Squared for Model Comparison

Question 1

You add two variables to a regression model. R² rises from 0.72 to 0.73, but adjusted R² falls from 0.71 to 0.70. What should you conclude?

• A. Keep both variables — R² increased, confirming they improve the model
• B. Drop both variables — the degrees-of-freedom penalty outweighs the variance they explain, so adjusted R² correctly indicates the model is worse
• C. The model is overfit and should be re-estimated on a holdout sample
• D. Adjusted R² is unreliable when R² increases, so R² should take priority

Answer: (B)

When adjusted R² falls while R² rises, the variables are not explaining enough new variance to justify the cost of the additional degrees of freedom. The adjusted formula penalizes each added variable by inflating the ratio RSS/(n−k−1); if the reduction in RSS is small, the penalty dominates and adjusted R² falls. This is exactly what adjusted R² is designed to detect — variables that appear to help (raising R²) but actually reduce the model's explanatory power per parameter. R² always weakly increases when variables are added, which is why it cannot be used for this comparison.

Question 2

Why can R² never decrease when you add another regressor to an OLS model?

• A. Adding a regressor increases the sample size, which mechanically improves fit
• B. OLS minimizes RSS, so the new coefficient is chosen to reduce RSS as much as possible — in the worst case it is set to zero, leaving RSS unchanged
• C. R² is normalized so that it is bounded below by the value from the smaller model
• D. Adding a variable always improves the model because OLS is unbiased

Answer: (B)

OLS finds the coefficient values that minimize RSS. When you add a new variable, OLS can always choose to set its coefficient to zero — reproducing the original model's RSS exactly. If any non-zero coefficient reduces RSS further, OLS will use it. So RSS can only stay the same or decrease, and R² = 1 − RSS/TSS can only stay the same or increase. This mechanical property means R² rewards model size regardless of whether the added variables are meaningful — the key flaw that adjusted R² corrects.

Question 3

Adjusted R² is typically between 0 and 1, just like ordinary R².

• T. True
• F. False

Answer: False

Adjusted R² can be negative. When the model explains almost nothing (RSS is close to TSS) and k is large, the degrees-of-freedom penalty can dominate, making the ratio RSS/(n−k−1) larger than TSS/(n−1), so adjusted R² = 1 − (that ratio) drops below zero. This is a signal that the model is essentially useless — it explains less variance per parameter than a model with just an intercept. Ordinary R² is bounded [0,1] because without the degrees-of-freedom correction, RSS ≤ TSS always.

Question 4

If adjusted R² decreases when a new variable is added, it means the variable explains less additional variance than the cost of the degree of freedom it consumes.

• T. True
• F. False

Answer: True

This is exactly the interpretation of the adjusted R² penalty. Each additional variable costs one degree of freedom, which inflates RSS/(n−k−1). If the variable reduces RSS substantially, this inflation is more than offset and adjusted R² rises. If the variable barely reduces RSS, the inflation dominates and adjusted R² falls. A falling adjusted R² is the signal that the variable is not earning its place in the model — a criterion that raw R² is incapable of detecting.

Question 5

A colleague argues 'R² is the right metric for model comparison because adding a useful variable always increases it.' What is wrong with this reasoning, and how does adjusted R² address the problem?

Model answer: The flaw is that R² increases for any added variable — useful or not — because OLS can always set the new coefficient to zero to avoid increasing RSS. A useless variable will still weakly increase R², making R² unable to distinguish between variables that genuinely improve the model and those that add noise. Adjusted R² addresses this by penalizing model complexity: it divides RSS by n−k−1 (shrinking as k increases), so adding a variable that barely reduces RSS will actually decrease adjusted R². Only variables that reduce RSS enough to offset the degrees-of-freedom cost will improve adjusted R².

The batting average analogy is helpful: R² is like total hits (always non-decreasing with more at-bats), while adjusted R² is like batting average (each new at-bat can hurt your average if you don't get a hit). This is why adjusted R² is used for comparing models with different numbers of regressors on the same dataset — it asks whether each variable is 'earning its keep.'