Questions: Qualitative Data Analysis and Thematic Coding

Question 1

A researcher interviews 20 participants about their experience with remote work. She notices that 18 out of 20 participants mention 'missing social interaction.' She names this a theme. A colleague argues this is not necessarily a theme. Who is right, and why?

• A. The researcher is right — frequency of occurrence is the defining criterion for a theme in thematic analysis
• B. The colleague has a point — a theme requires not just frequency but meaningful relevance to the research question and conceptual coherence
• C. The colleague is right — 18 out of 20 is actually too frequent to be a theme; themes should be patterns in a minority of the data
• D. Both are right — any pattern that appears in more than half the data automatically constitutes a theme

Answer: (B)

A theme in thematic analysis is not simply a frequently occurring topic; it captures something meaningful about the data in relation to the research question. 'Missing social interaction' might indeed be a theme — but its status as a theme depends on whether it illuminates something important about the experience of remote work, not just on how often it appears. Frequency is neither necessary nor sufficient: a rare but analytically significant pattern might be a theme; a ubiquitous mention that is tangential to the research question might not be.

Question 2

A research team has two coders independently code the same 50-page transcript and achieves only 60% agreement on code assignments. What is the most appropriate response?

• A. Average the two coders' interpretations to produce a combined dataset
• B. Accept the 60% agreement as sufficient since qualitative coding is inherently subjective
• C. Revisit and refine the operational definitions of ambiguous codes through discussion, then recode
• D. Have a third coder decide between the two interpretations wherever they disagree

Answer: (C)

Low inter-rater reliability signals that the code definitions are too ambiguous — coders are interpreting the same data differently because the codes lack clear operational definitions. The appropriate response is to examine the disagreements, discuss what each coder was noticing, and refine the code definitions to reduce ambiguity. This is directly analogous to refining operational definitions in quantitative research. Simply averaging interpretations (A) obscures the problem; accepting 60% (B) compromises rigor; a tie-breaking third coder (D) doesn't address the root cause.

Question 3

Qualitative research lacks rigor because it relies on the researcher's subjective interpretation of data rather than objective statistical analysis.

• T. True
• F. False

Answer: False

Qualitative research achieves rigor through systematic procedures, not statistical analysis. Inter-rater reliability, audit trails, member checking, and transparent documentation of analytic decisions are all tools for establishing trustworthiness — the qualitative analogue of reliability and validity. The fact that the researcher's perspective is involved does not make the analysis arbitrary; it requires reflexive rigor: being systematic and transparent about the analytic process. Qualitative and quantitative methods have different standards appropriate to their different goals, not a hierarchy of rigor.

Question 4

An audit trail in qualitative research documents analytic decisions so that others can evaluate the reasoning behind the analysis.

• T. True
• F. False

Answer: True

An audit trail is a detailed record of every significant analytic decision: why a code was created, why two codes were merged or split, why a particular theme was developed or dropped, how the analysis evolved across iterations. This transparency allows readers, reviewers, and other researchers to evaluate whether the conclusions are grounded in the data and whether the reasoning is sound — similar to how showing your work in a quantitative analysis allows others to check the statistical reasoning. Audit trails are a core trustworthiness strategy unique to qualitative research.

Question 5

Why is an audit trail important for establishing trustworthiness in qualitative research? What would be lost without it?

Model answer: An audit trail documents the researcher's analytic decisions throughout the analysis — why codes were created or merged, how themes were developed, what was considered and rejected. Without it, readers cannot distinguish between analysis that is grounded in systematic engagement with the data versus analysis that selectively represents only evidence that fits a preconceived interpretation. Trustworthiness in qualitative research depends on being able to evaluate the reasoning process, not just the conclusion. The audit trail makes that reasoning transparent and checkable, playing a similar role to showing statistical work in quantitative research.

The audit trail addresses a core vulnerability of qualitative analysis: because the researcher's judgment is central to the process, there is a risk that analysis reflects confirmation bias or selective attention. Systematic documentation externalizes the reasoning so it can be scrutinized. This is what separates rigorous qualitative analysis from mere opinion — the quality of the documented reasoning process, not the absence of human interpretation.