Extraordinary Claims Require Extraordinary Evidence

Graduate Depth 217 in the knowledge graph I know this Set as goal
bayesian evidence prior-probability sagan-standard

Core Idea

Carl Sagan's maxim is a direct consequence of Bayes' theorem. A claim with a very low prior probability requires evidence with a very high likelihood ratio to shift the posterior to a meaningful level. If your prior for a claim is 1 in a million, even evidence that is 100 times more likely under the hypothesis than under the alternative only brings the posterior to about 1 in 10,000 — still very unlikely. This is not a bias against unusual claims; it is a mathematical consequence of how evidence and priors interact. The practical lesson: calibrate the strength of evidence needed to the extremity of the claim, and be suspicious when extraordinary claims are supported only by ordinary evidence.

How It's Best Learned

Calculate concrete examples: if a friend claims to have seen a UFO (prior ~1 in 100,000 for an actual alien craft), how strong would the evidence need to be to make you believe? Work out the likelihood ratios. Compare with mundane claims ("it rained yesterday") where even modest evidence suffices because the prior is already high.

Common Misconceptions

Explainer

Carl Sagan's famous maxim -- "extraordinary claims require extraordinary evidence" -- sounds like a heuristic or a rhetorical device, but it is actually a direct mathematical consequence of Bayes' theorem. Understanding why makes the principle precise, quantitative, and much more useful than the slogan alone.

Bayesian updating works by multiplying your prior odds by the likelihood ratio of the evidence: posterior odds = prior odds x (P(evidence | claim true) / P(evidence | claim false)). A claim with a very low prior probability has very low prior odds. To bring the posterior to a meaningful level -- say, 50% -- the likelihood ratio must be large enough to compensate. If your prior for a claim is 1 in a million, you need evidence with a likelihood ratio of roughly a million to reach 50% posterior probability. Evidence with a likelihood ratio of 50 -- which would be decisive for a claim with moderate prior probability -- barely moves the needle, bringing a 1-in-a-million prior to only about 1 in 20,000. The math is unforgiving: the more extreme the prior, the stronger the evidence must be.

This is why the same type of evidence can be sufficient for one claim and woefully insufficient for another. If a friend tells you "it rained in Seattle yesterday," that testimony easily suffices -- the prior probability of rain in Seattle is high, and even modest evidence pushes you to near certainty. But if the same friend tells you they saw an alien spacecraft, their testimony carries the same likelihood ratio as before, yet it is applied to a prior of perhaps 1 in 100,000 for genuine alien craft. The result: the posterior barely budges. You would need evidence that is overwhelmingly more likely under "real alien craft" than under all mundane explanations combined -- multiple independent sensors, physical artifacts, convergent testimony from unconnected witnesses -- to produce the enormous likelihood ratio that the low prior demands.

Crucially, the Sagan standard does not mean extraordinary claims should be dismissed without investigation. It specifies how strong the evidence must be, not that evidence should not be sought. A scientist who refuses to investigate an extraordinary claim because "the prior is too low" has misunderstood Bayesian reasoning just as badly as one who accepts it on weak evidence. The principle is a calibration tool: it tells you what to expect from the evidence before you commit to a conclusion, and it warns you when the evidence presented is orders of magnitude weaker than what the claim requires. Applied honestly, it is a guard against both credulity (accepting extraordinary claims on ordinary evidence) and closed-mindedness (refusing to update even when extraordinary evidence is presented).

What did you take from this?

Topics in reflective domains aren't scored by quiz answers. Read, reflect, and mark when you've thought it through.

Quiz me anyway →

Prerequisite Chain

Counting to 10Counting to 20Understanding ZeroThe Number ZeroCounting to FiveOne-to-One CorrespondenceCombining Small Groups Within 5Addition Within 10Addition Within 20Two-Digit Addition Without RegroupingTwo-Digit Addition with RegroupingAddition Within 100Repeated Addition as MultiplicationMultiplication Facts Within 100Division as Equal SharingDivision as Grouping (Measurement Division)Division: Grouping (Repeated Subtraction) ModelDivision: Fair Sharing ModelDivision as Equal SharingDivision as GroupingBasic Division FactsDivision Facts Within 100Two-Digit by One-Digit DivisionDivision with RemaindersRemainders and Quotients in DivisionDivision Word ProblemsIntroduction to Long DivisionFactors and MultiplesPrime and Composite NumbersEquivalent FractionsRelating Fractions and DecimalsDecimal Place ValueIntegers and the Number LineComparing and Ordering IntegersAbsolute ValueAdding IntegersSubtracting IntegersMultiplying IntegersDividing IntegersUnit RatesProportionsPercent ConceptConverting Between Fractions, Decimals, and PercentsOperations with Rational NumbersTwo-Step EquationsSolving Multi-Step EquationsEquations with Variables on Both SidesAngle Pairs: Complementary, Supplementary, and VerticalParallel Lines and TransversalsCorresponding AnglesAlternate Interior AnglesTriangle Angle Sum TheoremExterior Angle TheoremTriangle Inequality TheoremSimilar Triangles: AA SimilaritySimilar Triangles: SSS and SAS SimilarityProportions in Similar TrianglesRight Triangle Trigonometry IntroductionTrigonometric Ratios ReviewRadian MeasureConverting Between Degrees and RadiansThe Unit CircleGraphing Sine and CosineGraphing Tangent and Reciprocal Trigonometric FunctionsDerivatives of Trigonometric FunctionsAntiderivativesIterated Integrals and Fubini's TheoremDouble Integrals in Cartesian CoordinatesDouble Integrals over Rectangular RegionsDouble Integrals in Polar CoordinatesDouble Integrals: Definition and SetupIterated Integrals and Fubini's TheoremDouble Integrals over Rectangular RegionsDouble Integrals over General RegionsApplications of Double Integrals: Area, Mass, and MomentsTriple Integrals in Cartesian CoordinatesTriple Integrals in Cylindrical and Spherical CoordinatesChange of Variables and the Jacobian DeterminantApplications of Triple Integrals: Volume and MassVector Fields and Their RepresentationsLine Integrals of Vector FieldsGreen's TheoremSurface Integrals and Flux of Vector FieldsSurface Integrals and Flux of Vector FieldsDivergence Theorem: Flux and OutflowDivergence TheoremElectric FluxGauss's LawConductors in Electrostatic EquilibriumCapacitance and CapacitorsDielectricsDielectric Constant and Relative PermittivityElectric Field Inside Dielectric MaterialsDielectric Materials and PolarizationDielectric Susceptibility and PermittivityEnergy Density in Electric FieldsElectric Current and Current DensityElectrical Resistance and ResistivityOhm's Law and Circuit ElementsElectromotive Force (EMF) and BatteriesKirchhoff's Circuit Laws: Voltage and CurrentDC Circuit Network Analysis MethodsTransient Response in RC CircuitsRC CircuitsLC and RLC CircuitsAC Circuits: FundamentalsImpedance and ReactanceAC Power and ResonanceElectromagnetic WavesThe Electromagnetic SpectrumBlackbody Radiation and Planck's LawPhotoelectric EffectThe Photon: Light as QuantaCompton ScatteringWave-Particle Dualityde Broglie WavelengthHeisenberg Uncertainty PrincipleWavefunction and the Born RuleThe Schrödinger EquationState Vectors and WavefunctionsQuantum SuperpositionQuantum EntanglementBell Theorem and Bell InequalitiesPostulates of Quantum MechanicsScattering TheoryIntroduction to Scattering TheoryPartial Wave Analysis in ScatteringSpin Angular MomentumElectron Spin and Intrinsic Magnetic MomentStern-Gerlach Experiment: Spin Quantization and MeasurementElectron Diffraction and Matter Wave PropertiesDavisson-Germer Experiment: Crystal Diffraction of ElectronsElectron Diffraction and Matter Wave InterferenceWavefunctions and Probability Density InterpretationQuantum Superposition and Linear Combinations of StatesQuantum Operators and ObservablesCanonical Commutation Relations and UncertaintyHeisenberg Uncertainty Principle and Measurement LimitsTime-Independent Schrödinger Equation and EigenvaluesHydrogen Atom in Quantum MechanicsSpectral Lines and Energy TransitionsSelection Rules for Atomic TransitionsLS and jj Coupling Schemes in Multi-Electron AtomsPauli Exclusion Principle and Antisymmetric WavefunctionsElectron Configuration and the Aufbau PrincipleThe Periodic Table and Atomic Electronic StructureThe Periodic TableElectron ConfigurationPeriodic TrendsIonization EnergyIonic BondingLewis StructuresResonance Structures and Delocalized ElectronsResonance and Formal ChargeMolecular Polarity and Dipole MomentsIntermolecular ForcesStates of Matter and Phase Changes: Melting, Boiling, and SublimationGas Laws and the Ideal Gas EquationGas Stoichiometry and Volume-Volume CalculationsThermochemistry and EnthalpyHeat Capacity and CalorimetryEntropy and Molecular DisorderSpontaneity and ΔGEntropy and Gibbs Free EnergyChemical EquilibriumAcid-Base ChemistryOrganic Reaction Mechanisms and Arrow PushingSN2 Substitution ReactionsSN1 Substitution ReactionsE1 Elimination ReactionsAlcohols and Ethers: Structure, Properties, and NomenclatureReactions of AlcoholsAldehydes and Ketones: Structure and ReactivityNucleophilic Addition to Aldehydes and KetonesCarboxylic Acids and Their DerivativesNucleophilic Acyl SubstitutionAmines: Structure, Basicity, and ReactionsAmine Reactivity: Nucleophilicity and BasicityAmino Acid Structure and PropertiesPeptide Bonds and Polypeptide FormationProtein Primary StructureProtein Secondary StructureProtein Tertiary StructureIon Channels and Selective Permeability MechanismsSensory Receptor Transduction and AdaptationSensory Transduction and EncodingSensory Pathways OverviewVisual Processing PathwayThe Dorsal Stream and Action ControlDorsal Stream and Visuomotor ControlSpatial Attention and Posterior Parietal CortexPrefrontal-Parietal Attention Networks and ControlExecutive Control Networks and the Prefrontal CortexNeuroeconomics and Value ComputationNeural Mechanisms of Decision-MakingWorking Memory Neural CircuitsMemory Encoding and Levels of ProcessingSemantic Memory and Network ModelsMental Models in Understanding and ReasoningProblem Representation and Solution SearchExpert Cognition and Knowledge OrganizationSchemas and Knowledge OrganizationCognitive Biases and Judgment Under UncertaintyHeuristics in Judgment and Decision MakingBase-Rate Integration and Bayesian Reasoning in ProbabilityLogical Validity and Belief Bias in ReasoningFrequency Estimation and Metacognitive JudgmentOverconfidence and Metacognitive IllusionsCalibration TrainingReference Class ForecastingFermi EstimationExpected Value Decision-MakingSunk Cost Recognition and Rational QuittingNewcomb's ProblemCausal vs. Evidential Decision TheoryConservation of Expected EvidenceAbsence of Evidence Is Evidence of AbsenceExtraordinary Claims Require Extraordinary Evidence

Longest path: 218 steps · 1391 total prerequisite topics

Prerequisites (2)

Leads To (0)

No topics depend on this one yet.