Questions: Epistemic Properties and Metrics

Question 1

A belief system where every belief raises the probability of every other belief scores very high on coherence measures. What can we conclude about the reliability of this system?

• A. It is highly reliable, since mutual support among beliefs tracks truth
• B. It may be systematically false — coherence measures internal consistency, not accuracy
• C. It is calibrated, since calibration and coherence are closely related
• D. Its reliability cannot be assessed independently of coherence

Answer: (B)

Coherence measures the degree to which beliefs mutually support each other — it is a measure of internal consistency. A conspiracy theory can be extraordinarily coherent while being entirely disconnected from reality. This is the central trade-off: a belief system can maximize coherence while scoring very low on reliability (frequency of true outputs). Coherence is not evidence of truth; it is evidence of internal fit.

Question 2

What is the defining property of a 'strictly proper scoring rule' in epistemic utility theory?

• A. It assigns higher scores to beliefs that are more informationally rich
• B. It penalizes overconfidence more severely than underconfidence
• C. The strategy that maximizes expected score is to report one's true credences
• D. It rewards coherence by giving bonuses when beliefs mutually support each other

Answer: (C)

A strictly proper scoring rule is one where the expected-utility-maximizing strategy is to report your actual credences honestly — not to hedge, not to overstate confidence. The Brier score and logarithmic scoring rule are canonical examples. This property matters because it means the scoring rule cannot be 'gamed': a reasoner who tries to improve their score by misreporting their beliefs will do worse in expectation than one who reports honestly. This makes proper scoring rules ideal tools for evaluating epistemic states.

Question 3

A belief system can achieve high coherence while being systematically false.

• T. True
• F. False

Answer: True

True. Coherence measures how well beliefs mutually support each other — it is a probabilistic measure of internal consistency. A belief system organized around a false but internally consistent worldview (e.g., a well-developed conspiracy theory) can score very high on formal coherence measures. This reveals the key trade-off: coherentist and reliabilist ideals can conflict. Coherence is not a guarantee of truth-tracking; it reflects how well beliefs fit together, not how accurately they track the world.

Question 4

Maximizing the coherence of a belief system is sufficient to maximize its epistemic utility.

• T. True
• F. False

Answer: False

False. Epistemic utility functions combine multiple dimensions — accuracy, calibration, and informativeness — and coherence is only one component. Maximizing coherence can actually reduce reliability (since a highly coherent but insulated worldview resists revision from disconfirming evidence), and it can conflict with informativeness (a tightly focused, risky prediction is more informative but may sacrifice some internal coherence). Epistemic utility functions exist precisely because single-metric optimization produces distorted epistemic states.

Question 5

Why might maximizing coherence and maximizing reliability pull in opposite directions for a belief system?

Model answer: Coherence measures internal consistency — how much each belief raises the probability of the others. Reliability measures the frequency of true outputs. A system can increase coherence by insulating beliefs from disconfirming evidence, ensuring they mutually reinforce each other. But this insulation also cuts the belief system off from the external world, allowing false beliefs to persist as long as they fit together. Conversely, a reliable belief-forming process (like careful empirical observation) might generate individual beliefs that don't cohere neatly, since the world doesn't always deliver neat, mutually confirming information.

This trade-off is at the heart of the coherentist vs. reliabilist debate in epistemology. Coherentism makes internal fit the mark of good epistemic standing; reliabilism makes track record the mark. Formal epistemic utility theory shows that both matter, and that collapsing them into a single metric misses the real structure of epistemic evaluation. Proper scoring rules try to combine accuracy and calibration in a principled way, but even they must navigate the tension between informativeness (committing strongly to a prediction) and reliability (being right at the committed confidence level).