Gauss–Seidel & Jacobi

Why iterate instead of invert?

Direct methods (Gaussian elimination, LU) cost on the order of n³ operations and fill memory with intermediate numbers. But the systems engineering throws at you — a finite-difference temperature grid, a circuit with thousands of nodes — are large and sparse (mostly zeros). Iterative methods exploit that: each sweep is cheap, and you stop as soon as you're accurate enough. The 2-D conduction lab's relaxation solver is exactly this idea on a grid.

Jacobi vs. Gauss–Seidel vs. SOR

Solve each equation for its diagonal unknown: xᵢ = (bᵢ − Σⱼ≠ᵢ aᵢⱼxⱼ)/aᵢᵢ. Jacobi updates every unknown using only last sweep's values. Gauss–Seidel uses each freshly-computed value immediately, so it usually converges about twice as fast. SOR over-shoots each Gauss–Seidel correction by a factor ω (1 < ω < 2); tuned well, it's dramatically faster still.

When does it converge?

A sufficient guarantee is diagonal dominance: each diagonal entry at least as big as the sum of the other magnitudes in its row, |aᵢᵢ| ≥ Σⱼ≠ᵢ |aᵢⱼ|. Most physical systems (heat balances, resistor networks) are naturally dominant. Load the "Not dominant" preset and watch the iteration diverge — then notice that simply reordering the equations can sometimes restore dominance and rescue it. The convergence plot above tells the story at a glance: a straight line sloping down means success; sloping up means trouble.