A frequentist 95% confidence interval for a treatment effect is [2, 8]. A Bayesian 95% credible interval with a non-informative prior is [2.1, 7.9]. What is the key interpretive difference?
AThere is no meaningful difference — both intervals contain the true value with 95% probability
BThe confidence interval means: if we repeated the study many times, 95% of computed intervals would contain the true value. The credible interval means: given the observed data and prior, there is a 95% probability the true value lies in this interval
CThe credible interval is always narrower than the confidence interval
DThe confidence interval is for the data; the credible interval is for the parameter
This is the fundamental philosophical distinction. A frequentist confidence interval is a statement about the long-run performance of the procedure — any single interval either contains the true value or does not, with no probability attached. A Bayesian credible interval is a direct probability statement about the parameter: given the data and prior, the probability that the true value falls in the interval is 95%. The credible interval answers the question clinicians are actually asking. With non-informative priors, the two intervals are often numerically similar, but their interpretations differ fundamentally.
Question 2 Multiple Choice
A researcher uses a strong prior centered on a treatment effect of 0 (skeptical prior) in a Bayesian analysis. Critics argue this biases the results. Is this criticism valid?
AYes — any informative prior is biased and should never be used
BPartially — the prior shifts the posterior toward 0, which may be appropriate (incorporating healthy skepticism about treatment claims) or inappropriate (ignoring strong pre-existing evidence), depending on context. The choice of prior should be transparent and sensitivity-analyzed
CNo — the prior has no effect on the posterior when the sample size is large
DNo — Bayesian methods are immune to bias by construction
A skeptical prior is a deliberate choice to require strong data evidence before concluding a treatment works — analogous to the frequentist framework's conservative alpha level. It is appropriate when most treatments fail and prior evidence is weak. It is inappropriate when substantial previous evidence supports the effect. The key to valid Bayesian analysis is transparency (reporting the prior and its justification) and sensitivity analysis (showing how results change under different priors). If the data are overwhelming, the posterior will be dominated by the data regardless of the prior.
Question 3 True / False
Bayesian methods are particularly advantageous for clinical trial monitoring because they can compute the posterior probability that a treatment is effective at each interim analysis without requiring multiple testing corrections.
TTrue
FFalse
Answer: True
In frequentist interim monitoring, each look at the data inflates the Type I error rate, requiring alpha-spending functions or group sequential boundaries to maintain overall alpha. Bayesian methods do not have this problem because they update the posterior distribution continuously — each analysis simply produces a new posterior given all data so far. The posterior probability that the treatment is effective (e.g., P(treatment effect > 0 | data)) can be computed at any time without adjusting for the number of looks. This makes Bayesian methods natural for adaptive trial designs, where the decision to continue, stop, or modify the trial depends on accumulating evidence.
Question 4 Short Answer
Explain why Bayesian analysis is said to answer the question clinicians actually care about, and what makes this different from the frequentist answer.
Think about your answer, then reveal below.
Model answer: Clinicians want to know: given the data from this study, what is the probability that the treatment works? Bayesian analysis answers this directly through the posterior probability. Frequentist analysis answers a different question: if the treatment had no effect, what is the probability of observing data this extreme or more? The p-value is about the probability of the data given a hypothesis (no effect), not the probability of the hypothesis given the data. Clinicians naturally think in Bayesian terms — 'how confident should I be that this treatment helps?' — but frequentist output requires careful translation.
The inversion is subtle but consequential. P(data | H0) ≠ P(H0 | data). A p-value of 0.03 does not mean there is a 3% chance the null is true — it means there is a 3% chance of seeing data this extreme if the null were true. The posterior probability P(H1 | data) that clinicians want requires Bayes' theorem and a prior. This distinction is one of the most frequently misunderstood concepts in applied statistics.