Vector b is projected onto subspace W, giving proj_W(b). Why is proj_W(b) the closest point in W to b, rather than some other vector in W?
ABecause the projection formula minimizes the number of basis vectors used
BBecause the error vector b − proj_W(b) is orthogonal to W, so any other choice increases distance by the Pythagorean theorem
CBecause the inner products ⟨b, uᵢ⟩ are always non-negative, ensuring minimum distance
DBecause proj_W(b) lies in W, and all vectors in W are equidistant from b
The error vector e = b − proj_W(b) is perpendicular to W by construction. For any other w ∈ W, write b − w = e + (proj_W(b) − w). Since e ⊥ W and (proj_W(b) − w) ∈ W, these are orthogonal, and by the Pythagorean theorem: ‖b − w‖² = ‖e‖² + ‖proj_W(b) − w‖² ≥ ‖e‖². So any other w is at least as far from b as the projection.
Question 2 Multiple Choice
If {u₁, u₂} is an orthonormal basis for W and proj_W(b) = ⟨b,u₁⟩u₁ + ⟨b,u₂⟩u₂, what can we say about the vector b − proj_W(b)?
AIt lies in W, because it is a linear combination of u₁ and u₂
BIt is zero, because the projection formula accounts for all components of b
CIt lies in W⊥, perpendicular to every vector in W
DIts magnitude equals ‖b‖, because the projection preserves length
The error vector b − proj_W(b) is the part of b that the projection 'left behind.' For any basis vector uᵢ: ⟨b − proj_W(b), uᵢ⟩ = ⟨b, uᵢ⟩ − ⟨b, uᵢ⟩ = 0. Since the error is orthogonal to every basis vector of W, it is orthogonal to every vector in W — it lies in W⊥.
Question 3 True / False
The formula proj_W(b) = ⟨b,u₁⟩u₁ + ⟨b,u₂⟩u₂ gives the correct orthogonal projection for any basis {u₁, u₂} of W.
TTrue
FFalse
Answer: False
False. This formula works only when {u₁, u₂} is an orthonormal basis — each vector has unit length and they are mutually orthogonal. For a general (non-orthonormal) basis, the formula overcounts or undercounts contributions because the basis vectors are not independent in the inner-product sense. The correct formula for a non-orthonormal basis requires the Gram matrix (AᵀA)⁻¹Aᵀ — precisely the least-squares normal equation. This is why Gram-Schmidt, which converts any basis to an orthonormal one, is so useful.
Question 4 True / False
The orthogonal projection of b onto W typically lies strictly between b and the origin.
TTrue
FFalse
Answer: False
False. The projection can be anywhere in W — including at the origin (if b ⊥ W, the projection is 0) or at b itself (if b ∈ W, the projection is b). The projection minimizes distance from b to W, but that doesn't constrain where in W the projected point falls. For example, if b is perpendicular to the line W, proj_W(b) = 0.
Question 5 Short Answer
Why does the error vector b − proj_W(b) lie in W⊥? Explain using the projection formula.
Think about your answer, then reveal below.
Model answer: For each orthonormal basis vector uᵢ of W: ⟨b − proj_W(b), uᵢ⟩ = ⟨b, uᵢ⟩ − ⟨proj_W(b), uᵢ⟩ = ⟨b, uᵢ⟩ − ⟨b, uᵢ⟩ = 0. Since the error is orthogonal to every basis vector of W, it is orthogonal to every vector in W, so it lies in W⊥.
The projection formula is constructed to extract the W-component of b. What remains after subtracting this component must be orthogonal to W. This decomposition b = proj_W(b) + (b − proj_W(b)) into a W-part and a W⊥-part is the fundamental theorem of orthogonal projections, and it is why the projection is the closest point in W to b.