Divide-and-Conquer and the Master Theorem

College Depth 63 in the knowledge graph I know this Set as goal
Unlocks 123 downstream topics
master-theorem divide-and-conquer merge-sort recurrences algorithm-analysis

Core Idea

Divide-and-conquer algorithms split a size-n problem into a subproblems of size n/b and combine results in O(nᵈ) time, yielding the recurrence T(n) = aT(n/b) + O(nᵈ). The Master Theorem gives the asymptotic solution: T(n) = Θ(nᵈ log n) if a = bᵈ, Θ(nᵈ) if a < bᵈ, and Θ(n^(log_b a)) if a > bᵈ. This covers merge sort (a=2, b=2, d=1 → Θ(n log n)), binary search (a=1, b=2, d=0 → Θ(log n)), and Strassen's matrix multiplication algorithm. The theorem is the primary tool for analyzing recursive algorithms.

How It's Best Learned

Apply the Master Theorem to 5-6 concrete recurrences before studying its proof. Verify each case against the recursion tree: draw the tree, compute work at each level, and sum across levels. Derive merge sort's complexity both via the theorem and by directly expanding the recursion tree.

Common Misconceptions

Explainer

Divide-and-conquer algorithms work by reducing a problem of size n to a smaller problems, each of size n/b, solving them recursively, and then combining the results. The key insight is that this recursive structure directly produces a recurrence relation — exactly the kind you studied before. If combining takes O(nᵈ) time, the total work satisfies T(n) = aT(n/b) + O(nᵈ), where a is the number of subproblems, b is the reduction factor, and d is the exponent of the combine step.

The Master Theorem solves this recurrence by comparing the rate at which work accumulates across levels of the recursion tree. Picture the recursion tree: at the top level there is O(nᵈ) combine work; at the next level there are a subproblems each of size n/b, contributing a · O((n/b)ᵈ) = O(nᵈ · a/bᵈ) total work. The ratio a/bᵈ tells you how fast work grows or shrinks per level. If a/bᵈ < 1, work shrinks geometrically and the top level dominates: T(n) = Θ(nᵈ). If a/bᵈ > 1, work grows geometrically and the leaves dominate: there are n^(log_b a) leaves, giving T(n) = Θ(n^(log_b a)). If a/bᵈ = 1, every level contributes equally and there are log_b n levels: T(n) = Θ(nᵈ log n).

Merge sort perfectly illustrates the equal-work case. It splits into a=2 subproblems of size n/2 (so b=2), and merging takes O(n) time (d=1). Check: a = 2, bᵈ = 2¹ = 2, so a = bᵈ. The Master Theorem immediately gives T(n) = Θ(n log n). Binary search, by contrast, has a=1 subproblem of size n/2 and O(1) combine work (d=0). Here a=1 and bᵈ=1, so again a = bᵈ and T(n) = Θ(log n). Strassen's algorithm for matrix multiplication has a=7, b=2, d=2, so bᵈ=4 < 7=a, meaning the leaves dominate and T(n) = Θ(n^(log₂ 7)) ≈ Θ(n^2.81).

The Master Theorem is powerful because it converts case analysis on the recursion tree into a quick arithmetic comparison. But be aware of its limits: it only applies when the combine cost is exactly polynomial (O(nᵈ) with no extra logarithmic factors). If the combine cost is, say, O(n log n), the standard theorem does not apply and you need the Akra-Bazzi method or direct expansion. Before applying the theorem, always verify that a ≥ 1, b > 1, and d ≥ 0 — these conditions ensure the recursion is well-formed and the tree has the geometric structure the theorem exploits.

Practice Questions 5 questions

Prerequisite Chain

Counting to 10Counting to 20Understanding ZeroThe Number ZeroCounting to FiveOne-to-One CorrespondenceCombining Small Groups Within 5Addition Within 10Addition Within 20Two-Digit Addition Without RegroupingTwo-Digit Addition with RegroupingAddition Within 100Repeated Addition as MultiplicationMultiplication Facts Within 100Division as Equal SharingDivision as Grouping (Measurement Division)Division: Grouping (Repeated Subtraction) ModelDivision: Fair Sharing ModelDivision as Equal SharingDivision as GroupingBasic Division FactsDivision Facts Within 100Two-Digit by One-Digit DivisionDivision with RemaindersRemainders and Quotients in DivisionDivision Word ProblemsIntroduction to Long DivisionFactors and MultiplesPrime and Composite NumbersEquivalent FractionsRelating Fractions and DecimalsDecimal Place ValueIntegers and the Number LineOpposites and Additive InversesAbsolute ValueAdding IntegersSubtracting IntegersMultiplying IntegersDividing IntegersUnit RatesProportionsPercent ConceptConverting Between Fractions, Decimals, and PercentsOperations with Rational NumbersTwo-Step EquationsSolving Multi-Step EquationsEquations with Variables on Both SidesLiteral EquationsSlope-Intercept FormPoint-Slope FormWriting Linear EquationsParallel and Perpendicular Line SlopesGraphing Linear EquationsPiecewise FunctionsStep FunctionsComposition of FunctionsInverse FunctionsRadical Functions and GraphsRational ExponentsExponential Functions and GraphsLogarithms IntroductionBig-O Notation and Asymptotic AnalysisAlgorithm Analysis and Complexity ClassesDivide-and-Conquer and the Master Theorem

Longest path: 64 steps · 302 total prerequisite topics

Prerequisites (4)

Leads To (4)