A topic in the Open Knowledge Graph — a free, open map of 15,290 topics and the order to learn them in.
Pontryagin's Maximum Principle provides necessary conditions for optimal control of dynamic systems with state and input constraints. Unlike the infinite-horizon LQR (which assumes quadratic cost and free final state), Pontryagin methods apply to finite-horizon problems with arbitrary cost and hard constraints (e.g., u ∈ [u_min, u_max]). The method introduces costate (adjoint) variables λ(t) that represent the marginal cost of changes to each state variable; optimal control u*(t) maximizes the Hamiltonian H = L(x,u,t) + λᵀ f(x,u,t) over all admissible u(t) at each instant. The result is often bang-bang control (u at its limits) with potential switches at specified times determined by a switching function. Indirect methods solve the Two-Point Boundary Value Problem (TPBVP); direct methods discretize and solve as finite-dimensional optimization.
Solve the Brachistochrone problem (find the fastest path between two points under gravity) or a simple finite-horizon minimum-fuel or minimum-time control problem (e.g., a spacecraft transferring between orbits with finite thrust). Derive the Hamiltonian, write the costate dynamics, use the maximum principle to characterize optimal u(t), and solve numerically via direct methods (transcription + nonlinear programming) or indirect methods (TPBVP solver). Observe that the optimal control is often bang-bang (at bounds) and the costates encode how much the objective improves if you relax a state constraint.
From LQR, you've seen that optimal control can be computed systematically: minimize a quadratic cost subject to linear dynamics. LQR gives a feedback law K that is optimal for the specific cost structure and infinite horizon. But many real problems don't fit this mold: minimum-time (get to target as fast as possible), minimum-fuel (use least fuel regardless of time), or finite-horizon problems with hard constraints on states and inputs. Pontryagin's Maximum Principle is the mathematical framework for these cases.
The principle is stated as a *necessary condition* for optimality: if u*(t) is an optimal control, then there exists a costate (adjoint) trajectory λ(t) such that at every time instant, u* maximizes the Hamiltonian H(x, u, λ, t) = L(x, u, t) + λᵀ f(x, u, t), where L is the running cost and f is the dynamics. This is profound: you don't solve a global optimization problem over the entire trajectory; instead, you solve a pointwise maximization of H at each instant, subject to the constraint that the costate evolves backward according to λ̇ = −(∂H/∂x)ᵀ with a terminal condition λ(T) = ∇g(x(T)) (where g is terminal cost).
The costate λ is best understood as the marginal cost of the state: λᵢ(t) tells you how much the optimal cost would improve if you could increase state i at time t. In physical systems, this has clear interpretations: for a spacecraft, the costate of velocity is a velocity impulse's worth of fuel; for an inventory problem, the costate of inventory is the cost of storage. The backward evolution of costate (from terminal cost at T, flowing to initial cost at 0) propagates the future cost impact of current state changes.
Bang-bang control emerges naturally: if the cost is minimum-fuel ∫|u|dt, then H = |u| + λᵀf(x,u). Maximizing over u means choosing the u that makes H largest. If |u| is piecewise-linear (as it is for bounded actuation |u| ≤ u_max), the maximum occurs at the boundary: u = u_max or u_min. The switching function Φ(t) = λᵀb(x,t) (the costate's projection onto the control direction) determines the sign of u: when Φ > 0, set u = u_max (accelerate); when Φ < 0, set u = u_min (decelerate). Zeros of Φ are switching times. For minimum-time problems, bang-bang is always optimal. For quadratic cost (LQR), the optimal control is smooth feedback u = −Kx, not bang-bang, because the quadratic cost penalizes large u, discouraging extreme actions.
Solving optimal control problems via Pontryagin can be done two ways: Indirect methods numerically solve the resulting Two-Point Boundary Value Problem (state and costate boundary conditions at different ends), which is small but nonlinear and sensitive to initial guesses. Direct methods discretize the trajectory into N time steps, convert the dynamics constraints into algebraic equations, and solve the resulting finite-dimensional nonlinear program — no costate computation needed. Direct methods handle path constraints naturally (inequalities on state during the trajectory, not just terminal) and are more robust, but solve larger problems.
Modern practice increasingly uses direct methods with standard NLP solvers (SNOPT, IPOPT), which scale to high dimensions and complex constraints. Aerospace applications routinely solve low-thrust trajectory optimization (find the fuel-optimal path to Mars) and missile guidance (compute real-time optimal intercept trajectories) using direct methods. The conceptual insight remains: the Pontryagin costate λ* from the solution encodes sensitivity information, and can be used to assess the value of relaxing constraints or perturbing the objective without re-solving.
The limitation of Pontryagin methods is that necessary conditions are not sufficient — a solution satisfying the principle is a candidate for optimality but may be a local minimum, a saddle point, or even a local maximum. For convex problems (convex cost, convex constraints), necessity implies sufficiency. For nonconvex problems (most practical systems), you must check second-order conditions (conjugate points in indirect methods, or validation with direct method solutions) to ensure global optimality. In practice, you solve the problem multiple ways and compare: if the indirect method (TPBVP) and direct method (NLP) both converge to the same cost, confidence in optimality is high.
No topics depend on this one yet.