In one sweep of Gauss-Seidel on a 4×4 system, you have just computed x₁^(k+1) = 3.2 and x₂^(k+1) = 1.7. When computing x₃^(k+1), which values do you use for x₁ and x₂?
Ax₁^(k) and x₂^(k) — the previous iteration's values, to keep the sweep consistent
Bx₁^(k+1) = 3.2 and x₂^(k+1) = 1.7 — the freshly updated values
CThe average of old and new values for each variable
DEither old or new values — Gauss-Seidel allows either
This is the defining feature of Gauss-Seidel: once a variable is updated within a sweep, its new value is immediately used for all subsequent updates in that same sweep. When computing x₃, the already-updated x₁^(k+1) and x₂^(k+1) are used, not the old values. Jacobi would use x₁^(k) and x₂^(k) here. This 'use it as soon as you have it' approach is exactly why Gauss-Seidel typically converges faster.
Question 2 Multiple Choice
A team wants to implement an iterative solver on a GPU with thousands of parallel processing units. They compare Jacobi and Gauss-Seidel. Which is better suited for this architecture, and why?
AGauss-Seidel, because it converges faster and will therefore use fewer cores overall
BJacobi, because each update depends only on previous-iteration values and all updates can be computed simultaneously
CGauss-Seidel, because its sequential structure maps naturally to GPU thread ordering
DBoth are equally suited — parallelizability does not depend on the update strategy
Jacobi's update for x_i^(k+1) depends only on old values x^(k), which are all fixed at the start of a sweep. This means all n updates can be computed simultaneously with no dependencies — ideal for massive parallelism. Gauss-Seidel's update for x_i depends on freshly updated x_j for j < i, creating a sequential dependency chain. This is the core trade-off: Gauss-Seidel needs fewer iterations but cannot be parallelized within a sweep; Jacobi is slower per sweep but trivially parallel.
Question 3 True / False
If Gauss-Seidel converges for a given system, it will require more iterations than Jacobi to reach the same level of accuracy.
TTrue
FFalse
Answer: False
Gauss-Seidel typically converges *faster* than Jacobi — roughly half as many sweeps to reach the same accuracy — because each update incorporates the most current available information rather than stale previous-iteration values. The cost is sequential dependency within each sweep (no parallelism within a sweep), but in terms of iteration count, Gauss-Seidel is the more efficient method. The statement reverses the relationship between the two.
Question 4 True / False
For Gauss-Seidel to converge, the matrix A should be symmetric positive definite.
TTrue
FFalse
Answer: False
Diagonal dominance is the more general condition that guarantees Gauss-Seidel convergence. Symmetric positive definite (SPD) matrices are a special case where convergence is guaranteed, but SPD is not necessary. Many diagonally dominant systems that are not SPD also converge under Gauss-Seidel. Knowing the sufficient conditions matters: diagonal dominance and SPD are both sufficient; neither is strictly necessary in practice, but they are the standard convergence guarantees to cite.
Question 5 Short Answer
Explain in your own words why Gauss-Seidel typically converges faster than Jacobi, and what you give up in exchange.
Think about your answer, then reveal below.
Model answer: Gauss-Seidel uses updated values immediately within each sweep: once x₁ is updated, the new value is used when computing x₂, and so on. Every variable update benefits from all information computed so far in that sweep, incorporating more accurate estimates than Jacobi's 'use only old values' approach. The result is roughly half as many sweeps to converge. The trade-off is that each x_i update depends on the freshly updated x_j for j < i, so all updates must run sequentially — unlike Jacobi where all updates are independent and can run in parallel.
This trade-off is at the heart of iterative solver design: more information per iteration vs. more parallelism per iteration. For large systems on modern hardware, Jacobi's parallelizability can outweigh its slower per-sweep convergence. For sequential computation, Gauss-Seidel wins. Successive over-relaxation (SOR) extends Gauss-Seidel further by extrapolating each update with a factor ω, accelerating convergence even more.