Qualitative Impact Assessment Methods

Core Idea

Qualitative impact assessment documents how interventions change people's lives through in-depth interviews, focus groups, and participatory evaluation. Rather than experimental comparison groups, QIA uses narrative evidence and thick description to demonstrate change. Strengths include capturing unexpected outcomes, processes, and individual trajectories; weaknesses include causal ambiguity and selection bias. QIA is valuable in development, social enterprise, and community-based programs where outcomes are context-dependent and heterogeneous.

Explainer

From your ethnography prerequisite, you know that qualitative research captures depth, context, and meaning that surveys cannot reach. Qualitative impact assessment (QIA) applies that capacity to a specific evaluative question: did this intervention change people's lives, and if so, how? The challenge is that "impact" in social programs is rarely a clean before-after difference. A community health program, a microfinance initiative, a youth mentorship scheme — these produce outcomes that are heterogeneous (different for different people), contextually shaped (the same program works differently in different communities), and entangled with everything else happening in participants' lives. Randomized experiments are often infeasible or ethically problematic in these settings. QIA provides an alternative: document the changes people themselves perceive, understand the mechanisms through which those changes occurred, and tell a credible story about the program's contribution.

The core methods build directly on ethnographic technique. In-depth interviews explore individual trajectories — how was your life before? what happened after you joined the program? what changed and why? The interviewer probes for specifics, examples, and counterfactuals ("would this have happened without the program?"). Focus groups reveal collective perspectives and social norms around the intervention, capturing what participants believe is acceptable to report publicly and how shared understandings of change have formed. Participatory evaluation goes further, involving community members in designing evaluation questions and collecting data — this often surfaces unexpected outcomes the external evaluator would never have thought to ask about. Most Significant Change (MSC) is a specific QIA technique: you ask participants to select and narrate the single most important change the program produced for them, then work backward to understand the process.

The central methodological challenge in QIA is attribution: how do you know the program caused the change rather than something else? Unlike an RCT with a control group, QIA cannot rule out confounds by design. The response is contribution analysis — rather than claiming sole causation, you build a theory of change (what the program was supposed to do and how), gather evidence that the theory's predicted steps actually occurred in the predicted sequence, and present a plausible causal narrative. The standard is not proof of causation but a reasoned, evidence-based argument that the program *contributed* to observed changes. Selection bias remains a genuine limitation: participants who agree to interviews may be those most positively affected, and thick description of their stories cannot represent those who left, stayed silent, or were harmed.

QIA is most powerful when it complements quantitative evaluation rather than replacing it. Numbers can tell you how many people's incomes rose; QIA tells you which households were able to leverage that income for education, which were trapped by social obligations that redistributed gains to extended family, and which experienced the program as humiliating surveillance. These are not minor details — they are often the findings that determine whether a program should be scaled, redesigned, or abandoned. The skill is knowing which questions need numbers and which need stories, and building an evaluation that provides both.