Questions: Monte Carlo Methods and Importance Sampling

Question 1

A naive Monte Carlo implementation samples configurations uniformly at random and averages A(s) over all samples. For a system of N = 100 spins at a low temperature, this approach fails because:

• A. Computing A(s) for any individual spin configuration is computationally too expensive
• B. The partition function Z cannot be computed, so absolute probabilities cannot be assigned to states
• C. At low temperatures, nearly all 2^100 configurations have negligible Boltzmann weight — uniform sampling almost never lands on a thermodynamically relevant state, so the average never converges
• D. The algorithm cannot distinguish between degenerate states with equal energy

Answer: (C)

At low temperature, the Boltzmann distribution is sharply concentrated on a tiny fraction of all configurations (the low-energy ones). Uniform random sampling draws from all 2^100 ≈ 10^30 configurations equally, so the probability of landing on any configuration with significant Boltzmann weight is astronomically small. You would need a sample size comparable to 2^100 before the average converged — which is the exact computation you were trying to avoid. Importance sampling solves this by biasing the random walk toward the thermodynamically relevant region.

Question 2

In the Metropolis algorithm, a proposed spin flip that raises energy by ΔE > 0 is accepted with probability exp(-ΔE/k_B T) rather than always or never. This acceptance rule ensures:

• A. The system always moves toward lower-energy configurations, efficiently finding the ground state
• B. Higher-energy configurations are never visited, keeping the simulation near equilibrium throughout
• C. The random walk samples configurations in proportion to their Boltzmann weights, implementing importance sampling without ever computing Z
• D. The simulation terminates quickly by rejecting unfavorable configurations

Answer: (C)

The Metropolis acceptance rule is designed so that in equilibrium, the ratio of the rates of moving from state A to B and B to A equals exp(-(E_B - E_A)/k_B T) — the ratio of their Boltzmann weights. This property (detailed balance) guarantees that after thermalization, the fraction of time the walk spends in any configuration equals its Boltzmann weight. Crucially, the acceptance probability exp(-ΔE/k_B T) depends only on the energy difference ΔE, not on the absolute energies — so Z cancels and never needs to be computed. Options A and B are wrong: the algorithm does visit higher-energy states (with reduced probability), which is essential for exploring the full equilibrium distribution.

Question 3

Monte Carlo methods scale polynomially with system size rather than exponentially because importance sampling focuses computation on the small fraction of configurations with significant Boltzmann weight.

• T. True
• F. False

Answer: True

Direct enumeration requires visiting all 2^N configurations, which is exponential in N. Monte Carlo replaces this with a random walk that visits configurations in proportion to their Boltzmann weight. The number of samples needed for a given statistical precision depends on the variance of A(s) under the Boltzmann distribution — a property of the physics — not on the total number of configurations. For most systems, this leads to polynomial scaling with N, making otherwise intractable systems (3D Ising model, lattice field theories) computationally accessible.

Question 4

In a Monte Carlo simulation, consecutive states in the random walk are statistically independent, so the number of Monte Carlo steps needed for a given precision equals the number of independent measurements required.

• T. True
• F. False

Answer: False

Consecutive states in the Metropolis walk are correlated — each state is produced by a small modification of the previous one. The autocorrelation time τ is the number of steps before two states are approximately independent. The effective number of independent samples is (total steps) / (2τ), not (total steps). Near a phase transition, correlation lengths diverge and τ grows dramatically (critical slowing down), so far more steps are required than the naive count would suggest. This is a practical bottleneck and motivates advanced algorithms like cluster updates that reduce τ.

Question 5

What is 'importance sampling' in Monte Carlo statistical mechanics, and why is it necessary for systems with large numbers of degrees of freedom?

Model answer: Importance sampling means generating random configurations according to the Boltzmann probability distribution P(s) ∝ exp(-E(s)/k_B T) rather than uniformly. When samples are drawn from P(s), the sample average of any observable A converges to the true thermal average ⟨A⟩ = Σ A(s)P(s). It is necessary because for large systems, the thermodynamically relevant configurations (those with significant Boltzmann weight) constitute an exponentially small fraction of all possible configurations. Uniform sampling would almost never land on a relevant state. By concentrating samples where the probability mass actually is, importance sampling makes the average converge with a feasible number of samples.

The key insight is that the Metropolis acceptance rule implements importance sampling without requiring explicit computation of Z. By accepting moves with probability min(1, exp(-ΔE/k_B T)), the walk naturally spends more time in low-energy states at low temperature, automatically biasing toward the thermodynamically relevant region.