A Lie group is a group that is also a smooth manifold — its elements can vary continuously and the group operations are smooth. Representations of Lie groups are required to be continuous (or smooth) homomorphisms G → GL(V), which is a much stronger constraint than for finite groups. For compact Lie groups (like SO(n), SU(n), U(n)), a complete analogue of finite group theory holds: every representation is completely reducible, characters determine representations, and orthogonality relations hold — with integrals over the group (via Haar measure) replacing finite sums.
Lie groups are groups with a manifold structure — they have both algebraic and geometric properties. Examples include GL_n(ℝ) (invertible n×n real matrices), SO(n) (rotations in n dimensions), SU(n) (special unitary matrices), and the circle group U(1) ≅ S¹. A representation of a Lie group is a continuous (equivalently, smooth) homomorphism ρ: G → GL(V). The continuity requirement is the key difference from finite group theory: it eliminates pathological homomorphisms and connects the representation theory to the group's geometry.
The exponential map exp: 𝔤 → G connects the Lie algebra 𝔤 (the tangent space at the identity, with Lie bracket [X,Y] = XY − YX for matrix groups) to the Lie group. A representation ρ: G → GL(V) induces a Lie algebra representation dρ: 𝔤 → 𝔤𝔩(V) defined by dρ(X) = (d/dt)|_{t=0} ρ(exp(tX)). For connected, simply connected groups, the representation theories of G and 𝔤 are equivalent — every Lie algebra representation integrates to a group representation. This reduces many questions to linear algebra on the Lie algebra.
For compact Lie groups, the theory parallels the finite group case remarkably closely. The Haar measure dg (the unique normalized left-invariant measure) replaces the counting measure (1/|G|)Σ. Maschke's theorem holds: every continuous finite-dimensional representation is completely reducible, via the same averaging argument with integrals replacing sums. Schur's lemma, orthogonality relations, and character theory all carry over. The irreducible representations are finite-dimensional and classified by highest weights — a combinatorial datum determined by the group's root system.
The Peter-Weyl theorem is the grand generalization. For a compact Lie group G, the matrix coefficients of all irreducible unitary representations form a complete orthonormal system in L²(G). The Hilbert space L²(G) decomposes as a completed direct sum: L²(G) ≅ ⊕̂ᵢ Vᵢ ⊗ Vᵢ*, where the sum runs over all irreducible representations. This is the infinite-dimensional analogue of ℂ[G] ≅ ⊕ Vᵢ^{dᵢ}. For non-compact groups (like SL_2(ℝ)), the theory is vastly more complicated: irreducible representations can be infinite-dimensional, complete reducibility fails, and the decomposition of L²(G) involves both discrete and continuous spectrum — the Plancherel formula replaces the Peter-Weyl theorem.