Questions: Simple (Bivariate) OLS Regression

Question 1

The OLS slope estimator β̂₁ in a bivariate regression of y on x is equal to which of the following?

• A. Var(x) / Cov(x, y)
• B. Cov(x, y) / Var(x)
• C. Cov(x, y) / Var(y)
• D. Corr(x, y) × Var(x)

Answer: (B)

β̂₁ = Cov(x, y) / Var(x). This formula comes directly from minimizing the sum of squared residuals: take the derivative of Σ(yᵢ − β₀ − β₁xᵢ)² with respect to β₁, set it to zero, and solve. The result says the slope equals how much x and y move together (covariance) normalized by how much x varies on its own (variance). If you flip numerator and denominator you get the inverse — not a valid slope estimator.

Question 2

A bivariate OLS regression with a large, statistically significant slope coefficient (β̂₁) necessarily implies that the model fits the data well.

• T. True
• F. False

Answer: False

The slope coefficient and model fit (R²) measure different things. β̂₁ captures the estimated relationship between x and y; R² measures what fraction of the total variation in y is explained by x. A large slope can coexist with a very low R² if there is enormous scatter around the regression line — for example, a steep average relationship between education and wages that has very high individual variability. Conversely, a tiny slope can yield a high R² if the data are very tightly clustered around the line.

Question 3

OLS estimates the line of best fit by minimizing a specific objective function. What does it minimize, and why is that criterion used rather than minimizing the sum of raw residuals?

Model answer: OLS minimizes the sum of squared residuals, Σ(yᵢ − ŷᵢ)². The sum of raw residuals is not used because positive and negative errors cancel out — any line through the mean of the data has a zero sum of residuals, so this criterion cannot distinguish a good fit from a bad one. Squaring the residuals eliminates cancellation, penalizes large errors more heavily than small ones, and produces a unique, analytically solvable minimum.

The cancellation problem is the key insight. If you simply sum (yᵢ − ŷᵢ), a line that systematically over-predicts half the data and under-predicts the other half by equal amounts scores the same as a line with no error at all. Squaring forces all contributions to be non-negative, so the only way to drive the sum toward zero is to make every individual residual small. This also explains why OLS is sensitive to outliers — a single point far from the line contributes a very large squared residual that the estimator works hard to reduce.