A skeptic's prior probability for claim X is 1 in 10,000. A witness reports seeing X, and this type of testimony is 50 times more likely when X is true than when X is false (likelihood ratio = 50). What is the approximate posterior probability that X is true after hearing the testimony?
AAbout 50% — a likelihood ratio of 50 makes the claim roughly equally likely to be true or false
BAbout 0.5% — the low prior still dominates; even strong evidence barely moves the needle on a 1-in-10,000 claim
CAbout 99% — a likelihood ratio of 50 is overwhelming evidence that almost always confirms the claim
DIt cannot be determined without knowing the base rate of the type of testimony
Using Bayes' theorem with prior odds of 1:9,999 and a likelihood ratio of 50: posterior odds = 50 × (1/9,999) ≈ 1:200, or about 0.5%. This is the mathematical heart of the Sagan standard. Even a likelihood ratio of 50 — which would be decisive for a claim with a moderate prior — barely moves the needle on a 1-in-10,000 claim. To reach 50% posterior probability from a 1-in-10,000 prior requires a likelihood ratio of approximately 10,000. The strength of evidence needed scales with the prior's extremity.
Question 2 Multiple Choice
You are evaluating two claims: (A) 'It rained in Seattle yesterday' (prior ~80%) and (B) 'A homeopathic remedy cured stage 4 cancer' (prior ~0.001%). A credible eyewitness report has the same likelihood ratio for both claims. For which claim does the eyewitness report more dramatically change your absolute probability estimate?
AClaim B, because any movement from near-zero requires proportionally larger updating
BBoth claims update by the same factor — the likelihood ratio is the same, so the multiplicative update is identical regardless of prior
CClaim A, because the absolute change in probability will be larger given the high prior
DNeither — eyewitness testimony has the same absolute effect regardless of the prior
The likelihood ratio multiplies the prior odds equally for both claims. But the *absolute* change in probability is much larger for Claim A because starting from 80%, a 50× likelihood ratio can move you to near certainty — a large absolute shift. Starting from 0.001%, the same 50× ratio only gets you to about 0.05% — a tiny absolute change. Claim A changes dramatically in absolute terms; Claim B barely moves. This is why ordinary evidence is sufficient for Claim A but woefully insufficient for Claim B — the prior's extremity sets how much evidence is needed to produce any meaningful absolute change.
Question 3 True / False
'Extraordinary claims require extraordinary evidence' means that claims with low prior probability should be rejected outright rather than investigated.
TTrue
FFalse
Answer: False
This is the most common misreading of the Sagan standard. The principle specifies how strong the evidence must be — it does not say skip the investigation. A 1-in-a-million prior claim can still become credible with sufficiently strong evidence (a likelihood ratio of millions). The principle is about calibrating the evidence threshold, not dismissing claims. A scientist who refuses to investigate an extraordinary claim because 'the prior is too low' has misunderstood Bayesian reasoning just as badly as the one who accepts it on weak evidence.
Question 4 True / False
'Extraordinary' in 'extraordinary claims' means claims that are unusual, shocking, or surprising to the person evaluating them.
TTrue
FFalse
Answer: False
The correct definition of 'extraordinary' here is objective, not subjective: a claim is extraordinary to the extent that it has a low prior probability given existing knowledge. A claim that is personally surprising to you might have a high prior in the reference class of well-informed people. Conversely, a mundane-sounding claim might be extraordinary if it conflicts with well-established science. 'Extraordinary' is not about emotional register — it is about location in probability space. This distinction matters because personal surprise is not a reliable guide to prior probability.
Question 5 Short Answer
Explain why the Sagan standard ('extraordinary claims require extraordinary evidence') is not just a heuristic or bias against novelty, but a direct mathematical consequence of Bayes' theorem.
Think about your answer, then reveal below.
Model answer: Bayesian updating works by multiplying prior odds by the likelihood ratio: posterior odds = prior odds × (P(evidence|claim true) / P(evidence|claim false)). A claim with prior probability 1 in N has prior odds of approximately 1:N. To reach even 50% posterior probability requires a likelihood ratio of approximately N. So a claim with prior 1 in a million mathematically requires evidence with a likelihood ratio of roughly a million — evidence that is a million times more likely if the claim is true than if it is false. This is not a preference or a conservative bias; it is what the numbers demand. The more extreme the prior, the larger the likelihood ratio must be to achieve any given posterior probability.
The power of this framing is that it transforms a vague intuition into a precise quantitative claim. Instead of 'I'm skeptical of UFOs,' the Bayesian says: 'My prior is 10^-6; show me evidence with a likelihood ratio of at least 10^4 and I'll take it seriously.' This makes the standard transparent and improvable — if you can establish a higher prior or demonstrate a higher likelihood ratio, you have made genuine progress on the question.