Numerical Stability and Conditioning

Graduate Depth 89 in the knowledge graph I know this Set as goal
Unlocks 6 downstream topics
stability conditioning error-analysis

Core Idea

A numerical algorithm is stable if small perturbations in inputs produce only small changes in outputs. Stability depends on both the problem (conditioning) and the algorithm (implementation). A well-conditioned problem solved with a stable algorithm yields accurate results; poor conditioning or instability can make even theoretically simple problems numerically unreliable.

Explainer

From your study of catastrophic cancellation, you saw a concrete failure mode: subtracting two nearly-equal floating-point numbers can wipe out all significant digits, turning a tiny relative error into a massive one. Numerical stability is the broader framework that explains when and why such failures occur, and how to reason about algorithm quality systematically.

The central distinction is between conditioning and stability. Conditioning is a property of the mathematical *problem* — how much does the true answer change when the inputs change slightly? Stability is a property of the *algorithm* — does the sequence of floating-point operations amplify errors, or keep them bounded? A well-conditioned problem has answers that are insensitive to small input perturbations. A stable algorithm is one that doesn't introduce unnecessary amplification beyond what the problem itself requires. These are independent: an unstable algorithm can ruin a well-conditioned problem, and a stable algorithm cannot rescue an ill-conditioned one.

Consider evaluating the polynomial p(x) = (x − 2)² near x = 2. Expanded as x² − 4x + 4, the three-term version subtracts large numbers that nearly cancel, risking catastrophic cancellation — unstable for this input. The factored form (x − 2)² avoids this cancellation entirely — mathematically identical, but numerically stable. This is a clean example where algorithm choice, not the problem, determines accuracy. Both formulas compute the same function; only their numerical behavior differs.

A useful benchmark concept is backward stability: an algorithm is backward stable if it computes the exact answer to a slightly perturbed problem. This framing separates algorithm error from problem sensitivity. If the backward error is tiny (the perturbed input is close to the real input) and the problem is well-conditioned (small input changes produce small output changes), then the computed answer is close to the true answer. Most reliable numerical algorithms — Gaussian elimination with pivoting, QR decomposition, stable ODE solvers — are designed with backward stability in mind. When you encounter numerical results you distrust, the first diagnostic question is always: is this problem inherently ill-conditioned, or is the algorithm introducing avoidable error?

Practice Questions 5 questions

Prerequisite Chain

Counting to 10Counting to 20Understanding ZeroThe Number ZeroCounting to FiveOne-to-One CorrespondenceCombining Small Groups Within 5Addition Within 10Addition Within 20Two-Digit Addition Without RegroupingTwo-Digit Addition with RegroupingAddition Within 100Repeated Addition as MultiplicationMultiplication Facts Within 100Division as Equal SharingDivision as Grouping (Measurement Division)Division: Grouping (Repeated Subtraction) ModelDivision: Fair Sharing ModelDivision as Equal SharingDivision as GroupingBasic Division FactsDivision Facts Within 100Two-Digit by One-Digit DivisionDivision with RemaindersRemainders and Quotients in DivisionDivision Word ProblemsIntroduction to Long DivisionFactors and MultiplesPrime and Composite NumbersEquivalent FractionsRelating Fractions and DecimalsDecimal Place ValueReading and Writing DecimalsComparing and Ordering DecimalsAdding and Subtracting DecimalsMultiplying DecimalsDividing DecimalsDividing FractionsMixed Number ArithmeticOrder of OperationsInteger Order of OperationsVariable ExpressionsCombining Like TermsOne-Step EquationsTwo-Step EquationsSolving Multi-Step EquationsEquations with Variables on Both SidesAngle Pairs: Complementary, Supplementary, and VerticalParallel Lines and TransversalsCorresponding AnglesAlternate Interior AnglesTriangle Angle Sum TheoremExterior Angle TheoremTriangle Inequality TheoremSimilar Triangles: AA SimilaritySimilar Triangles: SSS and SAS SimilarityProportions in Similar TrianglesRight Triangle Trigonometry IntroductionTrigonometric Ratios ReviewRadian MeasureConverting Between Degrees and RadiansThe Unit CircleGraphing Sine and CosineGraphing Tangent and Reciprocal Trigonometric FunctionsDerivatives of Trigonometric FunctionsAntiderivativesIndefinite IntegralsBasic Integration RulesRiemann SumsDefinite Integral DefinitionFundamental Theorem of Calculus Part 1Fundamental Theorem of Calculus Part 2U-SubstitutionIntegration by PartsSeparable Differential EquationsIntegrating Factor Method for First-Order Linear ODEsFirst-Order Linear Ordinary Differential EquationsSecond-Order Linear Homogeneous Differential EquationsCharacteristic Equation Method for Linear ODEsComplex Roots and Oscillatory SolutionsSpring-Mass Systems and Mechanical VibrationsResonance and Damping in Forced VibrationsRLC Circuit Applications of Differential EquationsIntroduction to Differential EquationsEuler's Method for Numerical SolutionsEuler's Method for ODEs (Error Analysis)Runge-Kutta MethodsStiff Differential Equations and Stability RegionsStability Regions and A-StabilityNumerical Stability and Conditioning

Longest path: 90 steps · 431 total prerequisite topics

Prerequisites (2)

Leads To (2)