Two floating-point numbers agree to their first 13 significant digits. You subtract them using a 15-significant-digit system. Approximately how many significant digits of accuracy does the result have?
A15 digits — subtraction does not affect the precision of either operand
B13 digits — precision is preserved because both inputs were accurate to 13 digits
CAbout 2 digits — the 13 shared leading digits cancel, leaving only the residual accuracy
DZero digits — the result is always exactly zero when numbers agree to this many digits
When two numbers agree to k significant digits, their difference has at most (15 − k) significant digits. Agreeing to 13 digits leaves only about 2 significant digits in the result. The leading 13 digits cancel away; what remains is dominated by rounding noise in both operands. Option D overstates the case — the numbers agree to 13 digits but not all 15, so the result is small but nonzero, just nearly all noise.
Question 2 Multiple Choice
You need to compute √(x² + 1) − √(x²) for x = 10⁷. The direct formula gives a result with almost no significant digits. What is the better approach?
AUse 128-bit floats to get more significant digits in the inputs
BMultiply and divide by the conjugate, rewriting as 1 / (√(x²+1) + √(x²))
CRound both square roots to fewer digits before subtracting to reduce cancellation
DCompute the two square roots separately and store them in distinct variables before subtracting
The conjugate reformulation avoids subtracting nearly-equal quantities entirely. The algebraically equivalent form 1 / (√(x²+1) + √(x²)) involves only addition in the denominator, which is numerically stable. Option A (higher precision) only shifts the danger threshold — for x = 10⁷, the operands agree to 7 digits, well within even double precision's range of cancellation. Options C and D change nothing about the structural cancellation problem.
Question 3 True / False
Catastrophic cancellation can be eliminated by switching from 32-bit (single) to 64-bit (double) precision arithmetic.
TTrue
FFalse
Answer: False
Higher precision reduces the relative error in each operand but cannot prevent those errors from dominating when the correct answer is much smaller than either operand. For the quadratic formula with b = 10⁸, cancellation occurs because operands agree to 8 digits — well within double precision's 15-digit range. Only algebraic reformulation changes the structure of the computation; higher precision only postpones the problem.
Question 4 True / False
Catastrophic cancellation can occur when two nearly equal floating-point numbers are added if they have opposite signs.
TTrue
FFalse
Answer: True
Adding numbers of opposite sign is equivalent to subtraction. If x = 1.000000000001 and y = −1.000000000000, then x + y involves the same near-cancellation as computing x − (−y) explicitly. The relevant criterion is whether the operands are nearly equal in magnitude and opposite in sign, not whether the operation is syntactically addition or subtraction. The sign and the magnitude together determine whether catastrophic cancellation occurs.
Question 5 Short Answer
Why does algebraic reformulation fix catastrophic cancellation when simply using higher precision does not?
Think about your answer, then reveal below.
Model answer: Higher precision reduces the relative error in each operand but cannot prevent those errors from dominating the result when the correct answer is orders of magnitude smaller than either operand. Reformulation restructures the computation so the cancellation-prone subtraction never occurs — you compute the same mathematical value via a different arithmetic path that avoids subtracting nearly-equal quantities. The problem is structural (the formula's shape and condition number), not representational (how many digits are used).
The quadratic formula example makes this concrete: using the conjugate form x = −2c / (b + √(b²−4ac)) for the small root avoids the cancellation regardless of precision, because the denominator involves addition rather than subtraction of nearly-equal terms. No amount of extra precision fixes a formula that is structurally ill-conditioned; only reformulation changes the condition number of the computation itself.