Questions: Evaluating Evidence in Inductive Arguments

Question 1

A polling organization surveys 10,000 people by calling landline telephone numbers and finds 65% support a policy. A student concludes: 'This is a very strong inductive argument — 10,000 is a huge sample.' What is the critical flaw in this reasoning?

• A. The student is correct; 10,000 is a reliable sample size for any population
• B. The sample is large but systematically biased — landline users skew older and wealthier, so the procedure misrepresents the general population regardless of sample size
• C. Telephone polling always produces weak evidence because people lie on the phone
• D. 65% support is too large a figure; any result above 60% should be treated with suspicion

Answer: (B)

This is the core insight: size cannot fix a biased sampling procedure. Landline-only polling systematically excludes mobile-only users, who tend to be younger, more urban, and less affluent — creating a directional error that grows larger with more data, not smaller. Representativeness is more fundamental than sample size. The law of large numbers reduces random error; it does nothing about systematic bias.

Question 2

Researchers find a strong positive correlation between time spent on social media and depression rates. To support the causal claim that social media causes depression, the most persuasive additional evidence would be:

• A. A larger study with 100,000 participants that replicates the same correlation
• B. Testimonials from psychiatrists who believe social media harms mental health
• C. A randomized experiment where participants are assigned to different social media usage levels, ruling out confounding variables
• D. A meta-analysis averaging results from 50 correlation studies

Answer: (C)

Correlation studies — no matter how large or how many — cannot rule out confounding variables such as pre-existing depression, personality traits, or socioeconomic factors. Only random assignment to conditions allows researchers to isolate the effect of social media from confounds. Options A and D add more correlation data but not causal evidence. Option B is anecdotal, not systematic evidence.

Question 3

A small but genuinely random sample can provide stronger inductive evidence than a large sample drawn from a biased sampling procedure.

• T. True
• F. False

Answer: True

Representativeness is more fundamental than size. A random sample distributes sampling errors randomly, so they tend to cancel out across observations and the sample converges on the true population value with more data. A biased sample consistently overrepresents some groups, and adding more data from the same biased procedure just reinforces the same directional error. Size matters for reducing random variation; only an unbiased procedure can eliminate systematic error.

Question 4

If a sample size is large enough, a biased sampling procedure will eventually produce representative results, because the law of large numbers guarantees convergence to the true population value.

• T. True
• F. False

Answer: False

The law of large numbers guarantees convergence to the true mean when samples are drawn randomly. It does not apply to systematically biased procedures. A phone survey that calls only landlines consistently excludes mobile-only users — adding more calls just collects more data from the same skewed pool. No amount of data from a broken procedure repairs the procedure. The distinction between random error (reducible by more data) and systematic bias (not reducible by more data) is fundamental to understanding when sample size matters.

Question 5

Explain the difference between random error and systematic bias in sampling, and why this distinction is fundamental to evaluating inductive evidence.

Model answer: Random error is variation that occurs by chance — any given sample might accidentally include slightly too many or too few members of a subgroup, but these errors vary in different directions across samples and tend to cancel out with more data. Systematic bias is a directional error built into the sampling procedure itself: it consistently overrepresents some groups and underrepresents others in the same direction every time. Because it is directional, it does not average out with larger samples. A phone survey that calls only landlines systematically excludes younger, more mobile-reliant populations no matter how many calls are made. This distinction matters because it determines what can fix the problem: random error is reduced by more data; systematic bias requires fixing the sampling procedure itself.

Students often assume that bigger samples are always better evidence. This is true only for random error. The harder and more practically important skill is recognizing when the sampling procedure itself introduces a directional distortion — which is the most common failure mode in real-world inductive arguments such as polls, studies, and surveys.