Likelihood Ratio Tests

Graduate Depth 86 in the knowledge graph I know this Set as goal
Unlocks 2 downstream topics
likelihood-ratio-tests hypothesis-testing statistics

Core Idea

The likelihood ratio test rejects H₀ when Λ = L(θ̂₀|X)/L(θ̂|X) < c, where θ̂₀ is the MLE under H₀ and θ̂ is the unrestricted MLE. Under H₀, -2log(Λ) converges in distribution to χ²_r where r is the dimension reduction. LR tests are general and achieve optimal Type II error (power) asymptotically.

Explainer

The Neyman-Pearson lemma — your core prerequisite — gave you the most powerful test for a specific kind of problem: a simple null hypothesis (H₀: θ = θ₀) against a simple alternative (H₁: θ = θ₁). The NP test rejects when the likelihood ratio L(θ₁|x)/L(θ₀|x) exceeds a threshold. That ratio compares two fixed parameter values. The likelihood ratio test generalizes this idea to composite hypotheses, where H₀ and H₁ each specify a set of parameter values rather than a single point.

The key insight is to replace the two fixed likelihoods with the best possible likelihoods under each hypothesis. Let Θ₀ be the null parameter space and Θ be the full parameter space. Define the likelihood ratio statistic Λ = sup_{θ ∈ Θ₀} L(θ|x) / sup_{θ ∈ Θ} L(θ|x). The numerator is the maximum likelihood achievable while respecting H₀; the denominator is the maximum likelihood overall, achieved at the unrestricted MLE θ̂. Since Θ₀ ⊆ Θ, we always have Λ ∈ [0, 1]. A value of Λ near 1 means the null hypothesis fits the data almost as well as the best unconstrained model — no reason to reject. A value of Λ near 0 means the data is far better explained by some θ outside Θ₀ — strong evidence against H₀. The test rejects when Λ < c for some threshold c.

The practical power of the LRT comes from Wilks' theorem: under H₀ and regularity conditions, the statistic −2 log Λ converges in distribution to a chi-squared distribution with r degrees of freedom, where r is the difference in the dimension of the full parameter space and the null parameter space (the number of constraints imposed by H₀). This asymptotic result means you can determine the critical value without knowing the exact distribution of Λ: just compare −2 log Λ to the χ²_r quantile for your chosen significance level. Your prerequisite on convergence in distribution is exactly what makes this work — you know that "converges in distribution to χ²_r" means the chi-squared approximation becomes exact as n → ∞, and is often good enough for moderate n.

As a concrete example, suppose X₁, …, Xₙ ~ Normal(μ, σ²) with both μ and σ² unknown, and you want to test H₀: μ = 0 against H₁: μ ≠ 0. The full model has two free parameters (μ, σ²); under H₀, only σ² is free. So r = 2 − 1 = 1, and −2 log Λ ≈ χ²₁. In this normal case, the LRT is equivalent to the t-test (the t-statistic squared follows an F-distribution, and by Wilks the LRT is asymptotically equivalent). For more complex models — exponential families, nested regression models, logistic regression — Wilks' theorem delivers the same chi-squared test, making the LRT a universal framework rather than a collection of special-case tests.

Practice Questions 5 questions

Prerequisite Chain

Counting to 10Counting to 20Understanding ZeroThe Number ZeroCounting to FiveOne-to-One CorrespondenceCombining Small Groups Within 5Addition Within 10Addition Within 20Two-Digit Addition Without RegroupingTwo-Digit Addition with RegroupingAddition Within 100Repeated Addition as MultiplicationMultiplication Facts Within 100Division as Equal SharingDivision as Grouping (Measurement Division)Division: Grouping (Repeated Subtraction) ModelDivision: Fair Sharing ModelDivision as Equal SharingDivision as GroupingBasic Division FactsDivision Facts Within 100Two-Digit by One-Digit DivisionDivision with RemaindersRemainders and Quotients in DivisionDivision Word ProblemsIntroduction to Long DivisionFactors and MultiplesPrime and Composite NumbersEquivalent FractionsRelating Fractions and DecimalsDecimal Place ValueReading and Writing DecimalsComparing and Ordering DecimalsAdding and Subtracting DecimalsMultiplying DecimalsDividing DecimalsDividing FractionsMixed Number ArithmeticOrder of OperationsInteger Order of OperationsVariable ExpressionsCombining Like TermsOne-Step EquationsTwo-Step EquationsSolving Multi-Step EquationsEquations with Variables on Both SidesAngle Pairs: Complementary, Supplementary, and VerticalParallel Lines and TransversalsCorresponding AnglesAlternate Interior AnglesTriangle Angle Sum TheoremExterior Angle TheoremTriangle Inequality TheoremSimilar Triangles: AA SimilaritySimilar Triangles: SSS and SAS SimilarityProportions in Similar TrianglesRight Triangle Trigonometry IntroductionTrigonometric Ratios ReviewRadian MeasureConverting Between Degrees and RadiansThe Unit CircleGraphing Sine and CosineGraphing Tangent and Reciprocal Trigonometric FunctionsDerivatives of Trigonometric FunctionsAntiderivativesIndefinite IntegralsBasic Integration RulesRiemann SumsDefinite Integral DefinitionFundamental Theorem of Calculus Part 1Fundamental Theorem of Calculus Part 2U-SubstitutionPartial Fraction Decomposition for IntegrationImproper Integrals - ConvergenceIntegral TestP-SeriesComparison TestLimit Comparison TestAbsolute vs. Conditional ConvergencePower SeriesTaylor PolynomialsTaylor SeriesMoment Generating FunctionsCharacteristic FunctionsConvergence in DistributionLikelihood Ratio Tests

Longest path: 87 steps · 475 total prerequisite topics

Prerequisites (2)

Leads To (1)