An LMS adaptive filter has been running stably. An engineer doubles the step size μ hoping to track faster environmental changes. What is the most likely trade-off?
AThe filter converges faster and achieves lower steady-state error simultaneously
BThe filter tracks changes faster but exhibits larger residual fluctuations (higher misadjustment) around the Wiener solution
CThe filter converges to a completely different optimal solution determined by μ
DThe filter's O(N) computational complexity per sample increases
Larger μ means bigger steps toward the Wiener solution, so adaptation is faster. But the instantaneous gradient estimate e[n]·x[n] is noisy, so large steps also produce larger random excursions around the optimal point — this is misadjustment. The Wiener solution being targeted doesn't change; only the precision of the final estimate does. There is always a speed-accuracy trade-off in LMS.
Question 2 Multiple Choice
Why does LMS not require explicit computation of the input autocorrelation matrix R_xx, unlike the Wiener filter?
ALMS only works on deterministic signals where R_xx is trivially the identity matrix
BLMS converges to a solution that is intentionally different from the Wiener optimal
CLMS uses the instantaneous product e[n]·x[n] as a noisy single-sample estimate of the gradient, avoiding any explicit statistical computation
DModern processors can invert R_xx fast enough that LMS skips the step for efficiency
The Wiener filter requires R_xx⁻¹ r_xd, which demands knowledge of the signal's second-order statistics — impractical when statistics are unknown or nonstationary. LMS replaces the true gradient −2·E[e[n]x[n]] with the instantaneous estimate −2·e[n]·x[n], a noisy but unbiased approximation. Because each update needs only the current error and input sample, R_xx is never computed.
Question 3 True / False
The LMS algorithm achieves O(N) computational complexity per update because it approximates the gradient using a single data sample rather than computing a full statistical expectation.
TTrue
FFalse
Answer: True
Each LMS update requires only one inner product (to compute the filter output ŷ[n] = wᵀx[n]) and one outer product (to update weights: w[n+1] = w[n] + 2μe[n]x[n]). Both are O(N) operations. The Wiener solution requires inverting R_xx — an O(N³) operation — which LMS avoids entirely by working sample-by-sample.
Question 4 True / False
Once an LMS adaptive filter has converged, its weight vector is fixed permanently at the Wiener solution and ceases to update.
TTrue
FFalse
Answer: False
LMS never stops updating — it applies a new gradient step at every sample. Because the gradient estimate is noisy, the weights continue to fluctuate randomly around the Wiener solution even after convergence. This residual fluctuation is called misadjustment and is the price paid for O(N) complexity. It is controlled by choosing a sufficiently small μ.
Question 5 Short Answer
The LMS algorithm uses a noisy, single-sample gradient estimate rather than the true gradient. Why does this still cause the weight vector to converge toward the Wiener solution over time?
Think about your answer, then reveal below.
Model answer: The instantaneous gradient estimate e[n]·x[n] is an unbiased estimate of the true gradient E[e[n]x[n]] — the noise averages to zero over many samples. Each individual step may point in a slightly wrong direction, but on average the steps point toward the Wiener solution. Over many iterations, the random errors cancel and the weight vector drifts in the correct direction in expectation, converging to the Wiener solution (with residual misadjustment proportional to μ).
This is the stochastic gradient descent principle: noisy gradient estimates work because their expectation equals the true gradient. The key condition is that the noise is zero-mean, so it neither biases the direction of convergence nor pushes the filter away from the Wiener solution on average.