A scientist finds that changing the last digit of one entry in b causes the solution x to change drastically. Their colleague suggests switching from Gaussian elimination to a more sophisticated iterative solver. Will this help?
AYes — iterative solvers are more numerically stable and will produce more accurate results
BNo — the sensitivity of x to perturbations in b is determined by κ(A), a property of the matrix, not the algorithm
CYes — but only if the iterative method also uses higher-precision floating point
DNo — the only fix is to eliminate the measurement error in b, after which any algorithm works
The condition number κ(A) measures how much the problem amplifies errors, not how much the algorithm does. A large κ(A) means that even tiny changes in b can produce large changes in x — this is true regardless of which algorithm is used. Switching solvers can improve stability (reduce algorithm-introduced rounding errors), but it cannot make the solution less sensitive to perturbations in the data. The problem is intrinsically ill-conditioned; no algorithm can extract information that isn't there.
Question 2 Multiple Choice
A matrix A has condition number κ(A) = 10⁸. If you solve Ax = b using double-precision arithmetic (about 16 significant decimal digits), approximately how many accurate significant digits can you expect in your solution?
A16 — double-precision always delivers 16 accurate digits regardless of the matrix
C0 — any condition number above 1 renders the solution meaningless
DIt depends on the algorithm; partial pivoting can recover all 16 digits
The condition number bounds relative error in x by κ(A) × (relative error in data). Double precision represents numbers to about 16 decimal digits, so floating-point round-off introduces relative errors of about 10⁻¹⁶. Multiplied by κ(A) = 10⁸, this gives up to 10⁻⁸ relative error in x — about 8 accurate digits. With κ(A) = 10¹⁶ on a 16-digit machine, the entire solution would be meaningless. This loss is intrinsic to the problem, not to the algorithm.
Question 3 True / False
An orthogonal matrix Q (where Q^T Q = I) has condition number κ(Q) = 1, making it perfectly conditioned.
TTrue
FFalse
Answer: True
An orthogonal matrix preserves vector lengths: ||Qx|| = ||x|| for all x. This means ||Q|| = 1 and ||Q⁻¹|| = ||Q^T|| = 1, so κ(Q) = ||Q|| · ||Q⁻¹|| = 1. Geometrically, Q only rotates and reflects the unit sphere — it maps it to itself, not to an elongated ellipsoid. A system Qx = b is maximally well-conditioned: perturbations in b produce equally-sized perturbations in x, with no amplification at all.
Question 4 True / False
Gaussian elimination with partial pivoting can reduce the condition number of an ill-conditioned matrix, improving how accurately the solution reflects the true answer.
TTrue
FFalse
Answer: False
Partial pivoting improves the numerical stability of the elimination algorithm — it prevents unnecessary amplification of rounding errors introduced during computation. But pivoting does not change the condition number, which is a property of the matrix A itself, not the computation. If A is ill-conditioned, the underlying problem is ill-conditioned regardless of how you solve it. Pivoting helps you reach the 'best possible' numerical answer given κ(A), but that best possible answer may still be highly inaccurate if κ(A) is large.
Question 5 Short Answer
Explain in your own words why an ill-conditioned system cannot be 'fixed' by using a better algorithm, and what the appropriate response is when you discover that κ(A) is very large.
Think about your answer, then reveal below.
Model answer: An ill-conditioned system is one where the data (A and b) do not contain enough information to determine x precisely — small uncertainties or errors in the data map to large uncertainties in the solution. This is a property of the problem, not the computation. No algorithm can extract precision that isn't present in the inputs. When κ(A) is large, the right responses are: question whether the problem is well-posed, look for sources of near-linear dependence in A, or apply regularization (e.g., Tikhonov regularization) which modifies the problem to trade sensitivity for stability.
This reflects the core principle: numerical algorithms solve mathematical problems; they cannot improve the mathematical problem itself. The condition number is the bridge between data quality (relative error in A and b) and solution quality (relative error in x). A condition number of 10¹² on a 16-digit machine means 12 digits are consumed by error amplification, leaving at most 4 meaningful digits — and no algorithm changes that arithmetic. Understanding this shifts the practitioner's attention from 'which solver?' to 'is this a well-posed problem?'