A topic in the Open Knowledge Graph — a free, open map of 15,290 topics and the order to learn them in.
Not all evidence is equal: evidence varies by type (anecdote, case study, observational study, randomized controlled trial, meta-analysis), by source reliability, and by its relevance to the specific claim. A hierarchy of evidence exists in empirical inquiry, with controlled experimental evidence generally outranking anecdote. Source evaluation requires examining: expertise of the source, potential conflicts of interest, methodology transparency, peer review, replicability, and whether the source reflects or diverges from expert consensus. Strong critical thinking requires calibrating confidence proportionally to the evidence.
Apply a structured source-evaluation rubric (CRAAP test or similar) to five sources on a contested empirical claim. Rank them by reliability and explain your ranking. Then discuss whether your initial intuitions about the sources were correct.
From inductive reasoning, you know that general conclusions are supported by evidence rather than guaranteed by it, and that the strength of an inductive argument depends on how much and how good the evidence is. Evidence evaluation is the skill of determining how good the evidence actually is — not just whether it exists, but whether it is reliable, representative, and relevant to the specific claim being made.
Evidence comes in a rough hierarchy of reliability. At the bottom sit anecdotes — single personal observations that are vivid but subject to selection bias, memory distortion, and confounding. A step up are case studies and expert testimony, which add domain knowledge but remain vulnerable to individual bias and lack of controls. Observational studies, which collect data across many cases without intervening, are more reliable still because patterns emerge across large samples. At the upper end sit randomized controlled trials (RCTs), where researchers randomly assign participants to conditions and control confounds directly. Higher still are meta-analyses — systematic aggregations of multiple RCTs that average out study-specific errors. Understanding this hierarchy is not about dismissing lower-tier evidence; anecdote can generate hypotheses, and case studies can reveal mechanisms. It is about calibrating how much confidence each type earns.
Source evaluation adds a second dimension: even high-quality evidence becomes unreliable if the source producing or reporting it is compromised. Your prerequisite on burden of proof gave you the principle that the person making a claim bears the burden of supporting it. Source evaluation helps you assess how well that burden is being met. Ask: Does the source have relevant expertise? Is there a conflict of interest — financial, ideological, or institutional — that could bias the conclusion? Is the methodology transparent and replicable? Has the work survived peer review and, better yet, independent replication? Does the source reflect or diverge from expert consensus, and if it diverges, what is the basis for the divergence?
The hardest skill here is calibrating confidence proportionally to evidence — neither dismissing weak evidence entirely nor over-crediting strong evidence as definitive. A single well-designed study does not settle a contested empirical question; it contributes one data point to a developing literature. Conversely, anecdotes from ten people you personally know do not override a well-replicated meta-analysis of 10,000 participants. This calibration requires accepting that your personal experience, however vivid, is a small and potentially unrepresentative sample of a much larger phenomenon.
The connection to your fallacy prerequisites is direct: the appeal to authority fallacy is not that citing experts is wrong — it is that citing experts without considering their expertise, methodology, or potential bias treats authority as a substitute for evidence rather than one indicator of reliability. Evaluating evidence is precisely the practice that distinguishes legitimate deference to expertise from uncritical authority-worship.
Topics in reflective domains aren't scored by quiz answers. Read, reflect, and mark when you've thought it through.