A program computes (1.0000001 − 1.0000000) in 64-bit floating-point. Both inputs are accurate to about 16 significant decimal digits, yet the result has far fewer significant digits. What phenomenon explains this?
AOverflow: the result exceeds the maximum representable floating-point value
BCatastrophic cancellation: subtracting nearly equal values destroys the leading significant digits, leaving only rounding-error residue
CUnderflow: the result is too small to be stored in normalized floating-point form
DTruncation error: the values were stored as integers and rounded before subtraction
When two nearly equal numbers are subtracted, their leading significant digits cancel. What remains is dominated by the rounding errors introduced when each value was originally stored. Even though each input had 16-digit accuracy, the result can have only a handful of correct digits — relative error explodes. This is catastrophic cancellation. It is not overflow (the values are small), underflow (the representability issue is about significant digits, not magnitude), or truncation error.
Question 2 Multiple Choice
An algorithm evaluates a function f whose condition number κ ≈ 10⁸. The inputs carry relative error ≈ 10⁻¹⁶ (one unit of machine epsilon). What relative error should you expect in the output?
AAbout 10⁻¹⁶, because machine epsilon bounds all floating-point errors regardless of the function
BAbout 10⁻⁸, because the condition number amplifies input relative error into output relative error
CAbout 10⁸, because the condition number is larger than machine epsilon
DExactly zero, because the algorithm uses exact arithmetic internally
The condition number κ measures how much relative error is amplified: output relative error ≈ κ × input relative error ≈ 10⁸ × 10⁻¹⁶ = 10⁻⁸. About 8 digits of precision are lost. This is why ill-conditioned problems (large κ) lose significant digits even with a perfect, stable algorithm — the sensitivity is in the problem, not the code.
Question 3 True / False
A numerically stable algorithm can still produce a result with large forward error if the problem itself is ill-conditioned.
TTrue
FFalse
Answer: True
Numerical stability (in Wilkinson's backward-error sense) means the algorithm returns the exact answer to a slightly perturbed input. If the problem is ill-conditioned, that nearby input may map to a very different output — so large forward error is possible even for a stable algorithm. Stability is a property of the algorithm; ill-conditioning is a property of the problem. Both matter for final accuracy, and they must be diagnosed separately.
Question 4 True / False
Floating-point rounding errors from chained operations tend to cancel out on average, so longer computations are generally as accurate as shorter ones.
TTrue
FFalse
Answer: False
Rounding errors do not systematically cancel — they accumulate. For n sequential operations, relative error can grow as O(n × ε_mach). More importantly, specific operations such as subtraction of nearly equal quantities cause catastrophic local amplification. The order and structure of operations affects accuracy dramatically, which is why numerical analysts reformulate algebraically equivalent expressions and why algorithms like Kahan compensated summation exist.
Question 5 Short Answer
What does it mean for an algorithm to be 'numerically stable' in terms of backward error analysis? Why is backward error analysis more useful than forward error analysis for judging algorithm quality?
Think about your answer, then reveal below.
Model answer: A numerically stable algorithm is one whose backward error is small — the computed output is the exact result of applying the algorithm to a slightly perturbed input, where the perturbation is comparable in size to machine epsilon. This separates algorithm behavior from problem difficulty: small backward error means any inaccuracy in the output is due to the problem's own ill-conditioning, not a flaw in the algorithm.
Forward error analysis tracks how rounding errors at each step accumulate to the final result — but it often wildly overestimates actual errors because errors of opposite sign partially cancel. Backward error analysis sidesteps this by asking: 'What input would have produced this output exactly?' If the answer is 'an input very close to the actual input,' the algorithm is well-behaved. This lets us certify algorithms like Gaussian elimination with partial pivoting as stable even when forward error bounds look alarming.