An evaluator assessing a microfinance program uses in-depth interviews and finds that participants credit significant life improvements to the program. A critic argues this proves nothing because there was no control group. The most methodologically defensible response is:
AAcknowledge that without a control group, QIA cannot make any causal claims and should be replaced with an RCT
BUse contribution analysis — build a theory of change, trace whether the predicted steps occurred in sequence, and make an evidence-based argument about the program's contribution to observed changes
CConduct retrospective matching to identify comparable non-participants for quasi-experimental comparison
DRestrict claims to descriptive findings and avoid any discussion of whether the program caused outcomes
QIA does not claim experimental proof of causation — it claims a reasoned, evidence-based argument for contribution. Contribution analysis asks: what was the theory of change? Did the predicted steps actually occur in the predicted sequence? Are alternative explanations plausible given the evidence? This is an honest response to the attribution challenge, not an evasion of it. Option A misunderstands QIA's appropriate standard of evidence; RCTs are often infeasible in complex social programs and would fail to answer the 'how and for whom' questions QIA is designed to address.
Question 2 Multiple Choice
Qualitative impact assessment is MOST valuable in which evaluation context?
AA program with clearly defined, uniform outcomes that need statistical significance testing across a large sample
BAn intervention whose outcomes are heterogeneous, context-dependent, and entangled with participants' broader life circumstances
CAn evaluation where a randomized controlled trial has already established causal impact and attribution is not in question
DA study requiring rapid turnaround at minimal cost with no deep engagement with participants
QIA's comparative advantages are depth, context-sensitivity, and capacity to surface unexpected outcomes. These matter most when outcomes vary substantially across individuals and settings, when the program's mechanisms are complex or poorly understood, and when the causal story needs to be understood (not just confirmed). When outcomes are uniform and measurable and a large sample is available, quantitative methods are usually better suited. QIA and quantitative evaluation are most powerful in combination: numbers show what changed; QIA shows why, for whom, and through what mechanisms.
Question 3 True / False
Qualitative impact assessment can demonstrate causation as rigorously as a randomized controlled trial, provided the qualitative data are sufficiently rich and the researcher conducts enough interviews.
TTrue
FFalse
Answer: False
No amount of qualitative richness eliminates the confounding that a control group addresses by design. An RCT achieves causal identification by randomly assigning participants, making the counterfactual explicit — what would have happened without the program. QIA has no equivalent mechanism; it can build a compelling contributory narrative and rule out some alternative explanations, but it cannot rule them all out with the same rigor. QIA's appropriate standard is contribution analysis — a reasoned, evidence-based argument — not experimental proof of causation.
Question 4 True / False
Selection bias is a genuine limitation of QIA because participants who agree to interviews may disproportionately represent those most positively affected by the program.
TTrue
FFalse
Answer: True
Participants who had negative or neutral experiences, who dropped out, who were harmed by the program, or who distrust the evaluator are less likely to participate in interviews. The resulting sample systematically overrepresents positive experiences, producing a skewed picture of impact. Good QIA practice includes actively seeking out non-participants, dropouts, and dissenters; using participatory methods that give communities control over data collection; and explicitly acknowledging whose voices are present and absent in the analysis.
Question 5 Short Answer
What is contribution analysis in QIA, and why does it represent an honest and defensible response to the attribution problem rather than an evasion of it?
Think about your answer, then reveal below.
Model answer: Contribution analysis begins with a theory of change — a specification of what the program was supposed to do and through what mechanisms. The analyst then gathers evidence that the theory's predicted steps actually occurred in the predicted sequence. If the causal pathway is visible in the data and alternative explanations are implausible given the evidence, the analyst can claim that the program contributed to observed changes — not that it was the sole cause. This is honest because it accurately represents what qualitative evidence can and cannot establish.
The key insight is that 'contribution' is a meaningful and useful standard in contexts where experimental proof is unavailable or inappropriate. Social programs almost never operate in isolation — other things are always happening in participants' lives. Attribution in this sense is not about sole causation but about a defensible claim that the program was a meaningful part of the causal story. A well-executed contribution analysis is more intellectually honest than either overclaiming causation (which the design cannot support) or refusing to make any causal claim (which renders evaluation uninformative).