Multiple Regression

College Depth 75 in the knowledge graph I know this Set as goal
Unlocks 113 downstream topics
multiple-regression OLS controls matrix-form

Core Idea

Multiple regression extends OLS to include several explanatory variables: y = β₀ + β₁x₁ + β₂x₂ + … + βₖxₖ + u. Each coefficient βⱼ represents the partial effect of xⱼ on y holding all other regressors constant — this 'ceteris paribus' interpretation is the central analytical payoff. In matrix form, the estimator is β̂ = (X'X)⁻¹X'y, which requires (X'X) to be invertible (no perfect multicollinearity). Adding control variables changes coefficient estimates if and only if those controls are correlated with both the dependent variable and the included regressors.

How It's Best Learned

Compare simple and multiple regression estimates on the same dataset — seeing how the wage coefficient on education changes when experience is added illustrates what 'holding constant' means in practice.

Common Misconceptions

Explainer

You already know bivariate regression: a single explanatory variable x₁ predicts y via ŷ = β̂₀ + β̂₁x₁, with OLS minimizing the sum of squared residuals. Multiple regression extends this to k explanatory variables — y = β₀ + β₁x₁ + β₂x₂ + … + βₖxₖ + u — and the conceptual payoff is enormous. Including additional regressors allows each coefficient to represent a partial effect: β₁ is the estimated change in y for a one-unit increase in x₁ *holding all other regressors constant*. This "ceteris paribus" interpretation is what lets economists isolate the effect of one variable from the confounding influence of others.

The wage-education example makes the logic concrete. A bivariate regression of wages on education gives a coefficient that captures not just education's direct effect but also any correlation between education and other determinants of wages (like experience or family background). When you add experience to the model, the education coefficient changes — and that change is informative. It tells you that part of the original estimate was actually attributable to the correlation between education and experience. The new coefficient is the effect of education among workers with the same years of experience.

In matrix notation, the OLS estimator is β̂ = (X'X)⁻¹X'y, where X is the n × (k+1) matrix of regressors including the constant column, and y is the n × 1 outcome vector. This formula generalizes the bivariate formula and makes the required conditions explicit: (X'X) must be invertible, which fails under perfect multicollinearity. You have seen matrix inverses in your prerequisites; here the condition det(X'X) ≠ 0 is the non-redundancy requirement — no regressor can be an exact linear combination of the others.

The "more controls is always better" intuition is wrong and important to resist. Adding a variable changes coefficient estimates only if it is correlated with both the outcome and the included regressors. Adding a truly irrelevant variable leaves coefficients unchanged in expectation but inflates their standard errors, reducing your ability to detect real effects. Adding an endogenous variable — one caused by your regressor — can introduce bias that wasn't there before, a phenomenon you'll study deeply when you reach omitted variable bias and simultaneity.

Multiple regression is the workhorse of empirical economics. From here, you'll study how to test whether a group of coefficients is jointly significant (F-tests), what happens when you omit a relevant variable, and how to handle categorical variables with dummies. Every one of those topics is an extension of the partial-effect logic you are building here.

Practice Questions 3 questions

Prerequisite Chain

Counting to 10Counting to 20Understanding ZeroThe Number ZeroCounting to FiveOne-to-One CorrespondenceCombining Small Groups Within 5Addition Within 10Addition Within 20Two-Digit Addition Without RegroupingTwo-Digit Addition with RegroupingAddition Within 100Repeated Addition as MultiplicationMultiplication Facts Within 100Division as Equal SharingDivision as Grouping (Measurement Division)Division: Grouping (Repeated Subtraction) ModelDivision: Fair Sharing ModelDivision as Equal SharingDivision as GroupingBasic Division FactsDivision Facts Within 100Two-Digit by One-Digit DivisionDivision with RemaindersRemainders and Quotients in DivisionDivision Word ProblemsIntroduction to Long DivisionFactors and MultiplesPrime and Composite NumbersEquivalent FractionsRelating Fractions and DecimalsDecimal Place ValueReading and Writing DecimalsComparing and Ordering DecimalsAdding and Subtracting DecimalsMultiplying DecimalsDividing DecimalsDividing FractionsMixed Number ArithmeticOrder of OperationsInteger Order of OperationsVariable ExpressionsCombining Like TermsOne-Step EquationsTwo-Step EquationsSolving Multi-Step EquationsEquations with Variables on Both SidesAngle Pairs: Complementary, Supplementary, and VerticalParallel Lines and TransversalsCorresponding AnglesAlternate Interior AnglesTriangle Angle Sum TheoremExterior Angle TheoremTriangle Inequality TheoremSimilar Triangles: AA SimilaritySimilar Triangles: SSS and SAS SimilarityProportions in Similar TrianglesRight Triangle Trigonometry IntroductionTrigonometric Ratios ReviewRadian MeasureConverting Between Degrees and RadiansThe Unit CircleGraphing Sine and CosineGraphing Tangent and Reciprocal Trigonometric FunctionsDerivatives of Trigonometric FunctionsAntiderivativesIndefinite IntegralsBasic Integration RulesRiemann SumsDefinite Integral DefinitionProbability Density Functions and Continuous DistributionsCumulative Distribution FunctionsContinuous Random VariablesNormal DistributionClassical OLS Assumptions (Gauss-Markov)Multiple Regression

Longest path: 76 steps · 399 total prerequisite topics

Prerequisites (8)

Leads To (19)