Questions: Selecting and Matching Research Designs to Questions

Question 1

A researcher wants to test whether a new tutoring program causes improvements in student test scores. Which design feature is logically necessary — not just helpful — to support a causal conclusion?

• A. A large sample size to increase statistical power
• B. Longitudinal follow-up to track students over multiple years
• C. Random assignment of students to tutoring versus control conditions
• D. Measuring both tutoring attendance and test scores in every student

Answer: (C)

Random assignment is logically necessary for causal inference because it makes the groups equivalent before the manipulation, ruling out the possibility that pre-existing differences explain the outcome. Without it, students who received tutoring might have been more motivated or better-resourced to begin with. Large sample size improves statistical precision but does not close the confound. Longitudinal follow-up and measuring both variables are useful but do not replace randomization as the mechanism that supports causality.

Question 2

A study finds that people who drink coffee daily have lower rates of Parkinson's disease. A journalist concludes that coffee prevents Parkinson's. What is the primary logical problem with this conclusion?

• A. The study did not measure Parkinson's disease accurately
• B. A third variable — such as overall lifestyle or genetic factors — might cause both higher coffee consumption and lower Parkinson's risk, explaining the correlation without any causal link
• C. The study did not follow participants long enough to observe Parkinson's onset
• D. The sample of coffee drinkers was probably too small to generalize

Answer: (B)

Correlational designs cannot rule out third-variable confounds. A lurking variable like an active lifestyle or genetic predisposition could independently predict both coffee drinking and Parkinson's risk, producing the observed correlation without any causal mechanism. The journalist's error is treating a correlational finding as causal — the study design simply cannot support that conclusion without manipulation and random assignment.

Question 3

A qualitative interview study cannot contribute to scientific understanding of causal mechanisms because it produces no quantifiable data.

• T. True
• F. False

Answer: False

Qualitative designs are well-suited for generating and refining causal theories by revealing the mechanisms and experiences underlying a phenomenon. While they cannot establish causation in the quantitative sense, they can surface plausible causal pathways that experiments can later test. The misconception conflates 'quantifiable' with 'scientific' — qualitative research can be rigorous, theory-informing, and essential for questions that quantitative designs would oversimplify.

Question 4

The right research design is determined primarily by the logical requirements of the research question — specifically, which alternative explanations must be closed to support the intended inference.

• T. True
• F. False

Answer: True

This is the core principle of design selection. A causal question requires closing third-variable confounds through manipulation and random assignment. A correlational question requires measuring both variables but not manipulating either. A phenomenological question requires rich qualitative data. The question type determines the design, not researcher preference, resource availability, or a general preference for complexity.

Question 5

Why is choosing the simplest design that logically supports the inference — rather than the most sophisticated one available — the correct standard for research design selection?

Model answer: More complex designs add cost, analytic difficulty, and ethical challenges without improving validity if they exceed what the question requires. The goal is to close the alternative explanations most relevant to the specific inference, not to close every possible alternative. A design should be matched to what the question logically demands: a correlational question does not benefit from random assignment because manipulation is not needed; a causal question cannot be answered by a correlational design regardless of its sophistication. Complexity beyond the minimum required actually reduces clarity and introduces unnecessary sources of error.

The key insight is that validity comes from the match between design logic and question type, not from complexity per se. Applying a randomized experiment to a 'how do people experience X?' question would suppress the rich, contextual data needed to answer it. Applying a correlational design to 'does X cause Y?' leaves confounds open that the question requires closing. The skill is identifying the minimum logical requirements of the question and selecting the simplest feasible design that meets them.