Questions: Condition Number of a Problem

Question 1

A numerical analyst uses a well-tested, numerically stable algorithm to compute ln(x) for x = 1.0001 and gets wildly inaccurate results. The most likely explanation is:

• A. The algorithm has a bug in its floating-point arithmetic
• B. The problem itself is ill-conditioned near x = 1: small relative errors in x cause enormous relative errors in ln(x)
• C. The computer lacks sufficient floating-point precision for any logarithm computation
• D. The algorithm should switch to double precision instead of single precision

Answer: (B)

Near x = 1, the condition number κ(x) = 1/|ln(x)| → ∞ as x → 1. This is a property of the mathematical problem, not the algorithm. Any algorithm — no matter how stable — must work with floating-point inputs that carry small relative errors, and the condition number tells you those errors will be amplified enormously. Switching precision or fixing the algorithm cannot rescue an ill-conditioned problem; the mathematics itself is sensitive.

Question 2

For f(x) = √x at x = 4, the relative condition number is κ = |x·f′(x)/f(x)| = |4·(1/4)/2| = 0.5. What does this mean?

• A. A 1% relative error in x produces approximately 0.5% relative error in √x — the problem is well-conditioned
• B. The algorithm for √x amplifies errors by a factor of 0.5 at this point
• C. √x is ill-conditioned at x = 4 because the condition number is less than 1
• D. A 1% relative error in x produces approximately 2% relative error in √x

Answer: (A)

The condition number is an amplification factor for relative errors. κ = 0.5 means a relative input error of ε produces a relative output error of at most 0.5ε — errors actually shrink. This is a well-conditioned problem. Option D inverts the factor; option B misattributes conditioning to the algorithm rather than the problem.

Question 3

Using a more numerically stable algorithm cannot reduce the output error below what the condition number predicts, given the precision of the input data.

• T. True
• F. False

Answer: True

The condition number sets the fundamental lower bound on achievable accuracy for a given input precision. A stable algorithm approaches this bound — it doesn't introduce unnecessary extra error — but cannot beat it. Conditioning is a problem property; stability is an algorithm property. The best possible outcome from any algorithm is output error ≈ κ × (input error).

Question 4

An ill-conditioned problem means the algorithm used to solve it is numerically unstable.

• T. True
• F. False

Answer: False

Conditioning describes the problem; stability describes the algorithm. They are independent properties. An ill-conditioned problem (large κ) will produce inaccurate results from any algorithm because tiny unavoidable floating-point input errors get amplified by κ — this has nothing to do with how the algorithm handles rounding internally. Conversely, a stable algorithm can solve a well-conditioned problem accurately, but no algorithm can fix an inherently ill-conditioned one.

Question 5

Why can a more numerically stable algorithm not 'fix' an ill-conditioned problem, even in principle?

Model answer: Because conditioning is a property of the mathematical problem, not the algorithm. The condition number quantifies how much relative error in the output is inherent for any given relative error in the input — it's a statement about the underlying mathematics. Floating-point inputs always carry small rounding errors, and the condition number says those errors will be amplified by a factor of κ regardless of how carefully the algorithm processes them. Stability only ensures an algorithm doesn't add *extra* amplification beyond what conditioning dictates; it cannot subtract the amplification the problem itself imposes.

The key conceptual move is separating problem sensitivity (conditioning) from algorithmic error propagation (stability). Once you accept that floating-point numbers always have small relative errors baked in, the condition number is the unavoidable tax those errors pay at the output. No computation can produce output more accurate than (input error) × κ for a problem with condition number κ.