← Graph View All Domains
Philosophy › Applied Rationality

Disagreement and Rational Updating

Graduate Depth 210 in the knowledge graph ☐ I know this ☆ Set as goal
epistemics disagreement aumann updating social-epistemology

Core Idea

Aumann's agreement theorem proves that two rational agents with common knowledge of each other's beliefs cannot agree to disagree — if they share the same priors and each knows the other's posterior, they must converge. In practice, persistent disagreement signals that at least one party has different priors, different evidence, or is reasoning incorrectly. The Rationalist approach to disagreement: take the other person's belief as evidence (their brain processed information you have not seen), update toward them proportional to your assessment of their reliability, and investigate the crux — the specific factual or inferential disagreement that drives the difference. Productive disagreement requires identifying cruxes rather than repeating arguments.

How It's Best Learned

In your next substantive disagreement, try to identify the crux: what is the specific factual claim or inference where you and the other person diverge? State it explicitly and check whether resolving that point would change both your minds. Practice taking the other person's confidence as evidence — if a domain expert disagrees with you, how much should you update?

Common Misconceptions

Explainer

From steelmanning, you know how to engage with the strongest version of an opposing argument. From intellectual humility, you know that calibrated confidence means being as certain as the evidence warrants, no more and no less. Disagreement and rational updating brings these skills to bear on one of the most common epistemic situations: someone you respect disagrees with you. What should you do?

The theoretical foundation is Aumann's agreement theorem, which proves that two rational agents with common priors and common knowledge of each other's posteriors cannot agree to disagree -- they must converge on the same probability estimate. The conditions are strict: shared priors, and each party knowing what the other believes (and knowing that the other knows, and so on). These conditions are rarely fully met in practice. But the theorem's practical lesson survives the idealization: persistent disagreement between reasonable people is a signal that something is going on -- different priors, different evidence, or reasoning errors on one or both sides. Treating disagreement as merely a difference of opinion misses the information it contains.

The Rationalist approach treats another person's belief as evidence. Their brain has processed information you have not seen, made inferences you have not made, and arrived at a conclusion that encodes all of that processing. When a well-calibrated domain expert tells you she estimates the probability of H at 30% and you had it at 70%, her estimate is a data point -- not as authoritative as a controlled experiment, but not as dismissible as mere opinion either. The rational response is to update toward her estimate by an amount proportional to your assessment of her reliability and relevant expertise. You do not blindly average (that ignores differences in information quality), you do not fully defer (that ignores your own information), and you certainly do not refuse to update (that treats your prior as immune to evidence).

The most productive move in any substantive disagreement is to identify the crux -- the specific factual claim, inference, or value judgment that actually drives the divergence. Most disagreements cycle endlessly because both parties restate their conclusions rather than locating the premise where they part ways. A crux-finding approach asks: "What would have to be true for you to update toward my view?" and "What would have to be true for me to update toward yours?" This locates the actual point of divergence, where evidence could potentially resolve the dispute. Once you have identified the crux, you can investigate it directly rather than rehearsing downstream disagreements that neither party will concede without the upstream issue being settled. This turns unproductive argument into collaborative inquiry -- which is what rational disagreement is supposed to be.

What did you take from this?

Topics in reflective domains aren't scored by quiz answers. Read, reflect, and mark when you've thought it through.

Quiz me anyway →

Prerequisite Chain

Counting to 10 → Counting to 20 → Understanding Zero → The Number Zero → Counting to Five → One-to-One Correspondence → Combining Small Groups Within 5 → Addition Within 10 → Addition Within 20 → Two-Digit Addition Without Regrouping → Two-Digit Addition with Regrouping → Addition Within 100 → Repeated Addition as Multiplication → Multiplication Facts Within 100 → Division as Equal Sharing → Division as Grouping (Measurement Division) → Division: Grouping (Repeated Subtraction) Model → Division: Fair Sharing Model → Division as Equal Sharing → Division as Grouping → Basic Division Facts → Division Facts Within 100 → Two-Digit by One-Digit Division → Division with Remainders → Remainders and Quotients in Division → Division Word Problems → Introduction to Long Division → Factors and Multiples → Prime and Composite Numbers → Equivalent Fractions → Relating Fractions and Decimals → Decimal Place Value → Integers and the Number Line → Comparing and Ordering Integers → Absolute Value → Adding Integers → Subtracting Integers → Multiplying Integers → Dividing Integers → Unit Rates → Proportions → Percent Concept → Converting Between Fractions, Decimals, and Percents → Operations with Rational Numbers → Two-Step Equations → Solving Multi-Step Equations → Equations with Variables on Both Sides → Angle Pairs: Complementary, Supplementary, and Vertical → Parallel Lines and Transversals → Corresponding Angles → Alternate Interior Angles → Triangle Angle Sum Theorem → Exterior Angle Theorem → Triangle Inequality Theorem → Similar Triangles: AA Similarity → Similar Triangles: SSS and SAS Similarity → Proportions in Similar Triangles → Right Triangle Trigonometry Introduction → Trigonometric Ratios Review → Radian Measure → Converting Between Degrees and Radians → The Unit Circle → Graphing Sine and Cosine → Graphing Tangent and Reciprocal Trigonometric Functions → Derivatives of Trigonometric Functions → Antiderivatives → Iterated Integrals and Fubini's Theorem → Double Integrals in Cartesian Coordinates → Double Integrals over Rectangular Regions → Double Integrals in Polar Coordinates → Double Integrals: Definition and Setup → Iterated Integrals and Fubini's Theorem → Double Integrals over Rectangular Regions → Double Integrals over General Regions → Applications of Double Integrals: Area, Mass, and Moments → Triple Integrals in Cartesian Coordinates → Triple Integrals in Cylindrical and Spherical Coordinates → Change of Variables and the Jacobian Determinant → Applications of Triple Integrals: Volume and Mass → Vector Fields and Their Representations → Line Integrals of Vector Fields → Green's Theorem → Surface Integrals and Flux of Vector Fields → Surface Integrals and Flux of Vector Fields → Divergence Theorem: Flux and Outflow → Divergence Theorem → Electric Flux → Gauss's Law → Conductors in Electrostatic Equilibrium → Capacitance and Capacitors → Dielectrics → Dielectric Constant and Relative Permittivity → Electric Field Inside Dielectric Materials → Dielectric Materials and Polarization → Dielectric Susceptibility and Permittivity → Energy Density in Electric Fields → Electric Current and Current Density → Electrical Resistance and Resistivity → Ohm's Law and Circuit Elements → Electromotive Force (EMF) and Batteries → Kirchhoff's Circuit Laws: Voltage and Current → DC Circuit Network Analysis Methods → Transient Response in RC Circuits → RC Circuits → LC and RLC Circuits → AC Circuits: Fundamentals → Impedance and Reactance → AC Power and Resonance → Electromagnetic Waves → The Electromagnetic Spectrum → Blackbody Radiation and Planck's Law → Photoelectric Effect → The Photon: Light as Quanta → Compton Scattering → Wave-Particle Duality → de Broglie Wavelength → Heisenberg Uncertainty Principle → Wavefunction and the Born Rule → The Schrödinger Equation → State Vectors and Wavefunctions → Quantum Superposition → Quantum Entanglement → Bell Theorem and Bell Inequalities → Postulates of Quantum Mechanics → Scattering Theory → Introduction to Scattering Theory → Partial Wave Analysis in Scattering → Spin Angular Momentum → Electron Spin and Intrinsic Magnetic Moment → Stern-Gerlach Experiment: Spin Quantization and Measurement → Electron Diffraction and Matter Wave Properties → Davisson-Germer Experiment: Crystal Diffraction of Electrons → Electron Diffraction and Matter Wave Interference → Wavefunctions and Probability Density Interpretation → Quantum Superposition and Linear Combinations of States → Quantum Operators and Observables → Canonical Commutation Relations and Uncertainty → Heisenberg Uncertainty Principle and Measurement Limits → Time-Independent Schrödinger Equation and Eigenvalues → Hydrogen Atom in Quantum Mechanics → Spectral Lines and Energy Transitions → Selection Rules for Atomic Transitions → LS and jj Coupling Schemes in Multi-Electron Atoms → Pauli Exclusion Principle and Antisymmetric Wavefunctions → Electron Configuration and the Aufbau Principle → The Periodic Table and Atomic Electronic Structure → The Periodic Table → Electron Configuration → Periodic Trends → Ionization Energy → Ionic Bonding → Lewis Structures → Resonance Structures and Delocalized Electrons → Resonance and Formal Charge → Molecular Polarity and Dipole Moments → Intermolecular Forces → States of Matter and Phase Changes: Melting, Boiling, and Sublimation → Gas Laws and the Ideal Gas Equation → Gas Stoichiometry and Volume-Volume Calculations → Thermochemistry and Enthalpy → Heat Capacity and Calorimetry → Entropy and Molecular Disorder → Spontaneity and ΔG → Entropy and Gibbs Free Energy → Chemical Equilibrium → Acid-Base Chemistry → Organic Reaction Mechanisms and Arrow Pushing → SN2 Substitution Reactions → SN1 Substitution Reactions → E1 Elimination Reactions → Alcohols and Ethers: Structure, Properties, and Nomenclature → Reactions of Alcohols → Aldehydes and Ketones: Structure and Reactivity → Nucleophilic Addition to Aldehydes and Ketones → Carboxylic Acids and Their Derivatives → Nucleophilic Acyl Substitution → Amines: Structure, Basicity, and Reactions → Amine Reactivity: Nucleophilicity and Basicity → Amino Acid Structure and Properties → Peptide Bonds and Polypeptide Formation → Protein Primary Structure → Protein Secondary Structure → Protein Tertiary Structure → Ion Channels and Selective Permeability Mechanisms → Sensory Receptor Transduction and Adaptation → Sensory Transduction and Encoding → Sensory Pathways Overview → Visual Processing Pathway → The Dorsal Stream and Action Control → Dorsal Stream and Visuomotor Control → Spatial Attention and Posterior Parietal Cortex → Prefrontal-Parietal Attention Networks and Control → Executive Control Networks and the Prefrontal Cortex → Neuroeconomics and Value Computation → Neural Mechanisms of Decision-Making → Working Memory Neural Circuits → Memory Encoding and Levels of Processing → Semantic Memory and Network Models → Mental Models in Understanding and Reasoning → Problem Representation and Solution Search → Expert Cognition and Knowledge Organization → Schemas and Knowledge Organization → Cognitive Biases and Judgment Under Uncertainty → Heuristics in Judgment and Decision Making → Base-Rate Integration and Bayesian Reasoning in Probability → Logical Validity and Belief Bias in Reasoning → Frequency Estimation and Metacognitive Judgment → Overconfidence and Metacognitive Illusions → Calibration Training → Intellectual Humility and Calibrated Uncertainty → Disagreement and Rational Updating

Longest path: 211 steps · 1371 total prerequisite topics

Prerequisites (3)

Steelmanninghard Intellectual Humility and Calibrated Uncertaintyhard Epistemology of Disagreementsoft

Leads To (0)

No topics depend on this one yet.