A linear system Ax = b is solved numerically. The right-hand side b had a relative error of 0.001% due to measurement noise, but the computed solution has a relative error of 10%. What does this tell you about the matrix A?
AThe algorithm has a bug — a correct algorithm would never amplify errors this much
BThe matrix A has a condition number of at least 10,000, meaning it is severely ill-conditioned and tiny perturbations in b can produce large errors in x
CThe matrix A must be singular, since no invertible matrix could amplify relative error by this factor
DThe measurement error in b must have been misreported — 10,000× amplification is physically impossible
The condition number κ(A) bounds the ratio of relative output error to relative input error. Here the amplification is 10%/0.001% = 10,000, so κ(A) ≥ 10,000 — a severely ill-conditioned system. A is invertible (it's not singular, and no bug is implied), but it nearly collapses some direction of space, and recovering from that near-collapse requires enormous amplification. The condition number is a property of the matrix, not of the algorithm. A perfect algorithm cannot do better than the condition number allows.
Question 2 Multiple Choice
Why is the condition number κ(A) = σ_max/σ_min, rather than just σ_max, the right measure of numerical difficulty for solving Ax = b?
ABecause σ_max alone tells you how large A can make vectors, but conditioning depends on both how much A stretches and how much it compresses — A⁻¹ must undo the compression, amplifying errors in directions with small singular values
BBecause σ_max can equal zero, making it an unreliable measure, while the ratio is always well-defined
CBecause Ax = b always involves only the minimum singular value; σ_max is irrelevant to error analysis
DBecause σ_min/σ_max is the condition number; the formula given has the ratio inverted
Solving Ax = b requires computing A⁻¹b. A⁻¹ has singular values 1/σ₁, ..., 1/σₙ — it compresses by σ_max and stretches by 1/σ_min. A perturbation δb with a component in the direction of the smallest singular value gets amplified by 1/σ_min. The overall worst-case amplification of relative error is ||A||·||A⁻¹|| = σ_max·(1/σ_min) = σ_max/σ_min. σ_max alone would only tell you how large A can make a vector — it misses the critical question of how much A⁻¹ amplifies noise in ill-conditioned directions.
Question 3 True / False
A matrix with condition number κ = 1 is the best-conditioned possible, meaning perturbations in b cause no amplification in the error in x.
TTrue
FFalse
Answer: True
True. κ(A) = 1 means σ_max = σ_min — all singular values are equal. Geometrically, the matrix stretches every direction by the same factor, so its inverse compresses every direction by the same factor. Perturbations in b are transformed but not selectively amplified in any direction. Orthogonal matrices (rotation/reflection matrices) have κ = 1 because they preserve all lengths. κ = 1 is the theoretical minimum; real-world problems aim for condition numbers small enough that errors stay within acceptable bounds.
Question 4 True / False
The condition number of a matrix depends on the specific right-hand side vector b — a different b in Ax = b leads to a different condition number.
TTrue
FFalse
Answer: False
False. The condition number κ(A) = ||A||·||A⁻¹|| is a property of the matrix A alone, independent of b. It is the worst-case ratio of relative output error to relative input error, maximized over all possible perturbation directions. While the actual error in a specific solution does depend on how b aligns with A's singular vectors, the condition number captures the worst possible sensitivity of the matrix as a structural property of A itself — not of any particular right-hand side.
Question 5 Short Answer
Explain in geometric terms why a matrix with σ_min ≈ 0 makes solving Ax = b numerically unreliable, even when b is known exactly.
Think about your answer, then reveal below.
Model answer: When σ_min ≈ 0, the matrix A nearly collapses space in some direction — it maps a nonzero vector to nearly zero in that direction. To solve Ax = b requires applying A⁻¹, which must undo this collapse: it stretches that near-zero direction back out by a factor of 1/σ_min ≈ ∞. Any tiny numerical error in b that has a component in this direction gets amplified enormously in the solution. The matrix cannot distinguish between the true b and a b contaminated by small errors, because A maps many different x vectors to nearly the same b — recovering x from b is inherently ambiguous.
This is the geometric meaning of ill-conditioning. A nearly singular matrix compresses some direction to near-zero — it is nearly non-injective. Inverting this compression requires enormous amplification. The condition number σ_max/σ_min is the ratio of most-stretched to most-compressed direction; when σ_min ≈ 0, this ratio is huge and any solver will suffer. The cure is regularization (adding a small δI to make the system well-conditioned) or reformulating the problem, not finding a better algorithm.