Questions: Computational Simulation of Social Systems

Question 1

A researcher builds a social simulation of opinion polarization with 50 free parameters. After extensive calibration, the model perfectly reproduces observed polarization trends in US politics over the past 20 years. What should we conclude?

• A. The model has identified the true causal mechanisms driving polarization and is ready to inform policy
• B. The model is behaviorally valid and this constitutes full validation for causal claims
• C. The perfect fit is expected and not very informative — with enough free parameters, almost any data pattern can be reproduced
• D. The model should now be tested by adding more agents to scale up its predictions

Answer: (C)

This is the core validation problem in social simulation. A sufficiently complex model with enough free parameters can be tuned post-hoc to fit almost any data pattern without capturing real mechanisms — this is curve-fitting, not causal inference. Full validation requires multiple strategies: face validity (do the mechanisms match domain knowledge?), structural validity (does the causal structure match theory?), behavioral validity (does the model reproduce patterns out-of-sample?), and predictive validity (does it forecast outcomes it wasn't calibrated to?). Perfect fit on training data is the weakest form of evidence.

Question 2

What is the key insight demonstrated by Schelling's segregation model?

• A. Racial segregation in cities requires active intentional discrimination by individual actors to be sustained
• B. Even mild preferences for same-type neighbors at the individual level can produce dramatic aggregate segregation that no agent intended
• C. The model proves that integration policies are ineffective because segregation is driven by preference
• D. Agent-based models can accurately reproduce the exact mechanisms of historical housing discrimination

Answer: (B)

Schelling's model is the canonical demonstration of emergence: macro-level patterns that arise from micro-level rules and that cannot be inferred by summing individual behaviors. Even agents with only mild same-neighbor preferences (not extreme prejudice) produce stark aggregate segregation. This is why simulation is powerful — it reveals how macro-level outcomes can emerge from micro-level rules in ways that are counterintuitive and impossible to detect by inspecting individual agents. The model shows that dramatic segregation doesn't require anyone to want dramatic segregation.

Question 3

A social simulation that successfully reproduces known empirical patterns — trends already observed in real data — is thereby validated as a causal model of the underlying processes.

• T. True
• F. False

Answer: False

Behavioral validity (reproducing known patterns) is one type of validation check, but it is not sufficient for causal validation. Many different causal models can produce identical observable patterns. Full validation requires structural validity (do internal mechanisms match theoretical claims about how the system works?), predictive validity (does the model forecast out-of-sample outcomes it wasn't calibrated to?), and ideally experimental corroboration. A model calibrated to fit known data may be entirely wrong about mechanisms while producing correct outputs within the training range.

Question 4

Agent-based models are better suited for modeling heterogeneous agents and spatial effects, while system dynamics is better suited for modeling feedback loops among aggregate quantities.

• T. True
• F. False

Answer: True

Different simulation tools have characteristic strengths that match different research questions. ABM models individual agents with distinct properties and local interactions, capturing spatial effects and heterogeneous behavior. System dynamics models aggregate stocks and flows using differential equations, capturing macro-level feedback without tracking individuals. Network simulation models diffusion and contagion through relational structures. Choosing the right approach requires matching the simulation architecture to the mechanisms you're trying to study.

Question 5

Why is 'the simulation reproduces the observed data' insufficient validation for a social simulation model, and what additional evidence would strengthen causal claims?

Model answer: Reproducing observed data is insufficient because a model with enough free parameters can fit almost any historical pattern through post-hoc tuning — demonstrating nothing about whether it captures real mechanisms. To strengthen causal claims: (1) structural validity should be assessed — do the model's internal mechanisms match empirical and theoretical knowledge of how the system works? (2) out-of-sample predictive validity should be tested — does the model accurately forecast outcomes it was not calibrated to? (3) ideally, the model's causal claims should be testable against natural experiments or field data. Pairing simulation with empirical evidence rather than treating it as a substitute for evidence is what distinguishes scientific modeling from sophisticated curve-fitting.

The practical implication is that simulation should be embedded in a research program that includes empirical data collection, experimental testing, and theoretical grounding — not used as a standalone tool for generating outputs that look like reality.