Demographic parity requires equal prediction rates (positive predictions) across protected groups. Why is this definition problematic?
Think about your answer, then reveal below.
Model answer: Demographic parity does not account for true differences in outcomes across groups. For example, if historical data shows that one group truly has higher qualifications for jobs, enforcing demographic parity (hiring the same percentage from each group) would hire unqualified candidates from one group, unfairly disadvantaging them and violating merit-based fairness. Additionally, demographic parity is incompatible with other fairness definitions (e.g., calibration) when base rates differ across groups. Demographic parity is appropriate when the underlying outcome should be independent of group membership, but inappropriate when there are legitimate outcome differences.
This highlights the fundamental tension in fairness: different definitions are appropriate for different contexts. The wrong definition can cause harm: enforcing demographic parity inappropriately can hurt the groups it intends to help by lowering standards.
Question 2 Multiple Choice
Equalized odds requires equal false positive and false negative rates across protected groups. How does this relate to fair misclassification?
AEqualized odds has no relationship to false positive/negative rates
BEqualized odds ensures that errors affect groups equally, so no group is systematically disadvantaged by misclassification
CEqualized odds only cares about false positives, not false negatives
DEqualized odds is the same as demographic parity
Equalized odds (also called 'equal opportunity' in some contexts) requires FPR and FNR to be equal across groups. This ensures that errors (false positives and false negatives) do not systematically disadvantage one group. For example, in hiring, false negatives (qualified candidates rejected) should occur at the same rate for each demographic group, and false positives (unqualified candidates hired) should also be equal. This makes the algorithm equally unreliable for all groups, ensuring fairness in error distribution. Unlike demographic parity, equalized odds allows different positive prediction rates if there are legitimate differences in underlying outcomes, making it more compatible with calibration.
Question 3 Multiple Choice
Calibration requires that predictions are equally accurate/reliable across protected groups. In what situation would calibration conflict with equalized odds?
ACalibration always agrees with equalized odds; they cannot conflict
BWhen base rates (true outcome rates) differ significantly across groups, satisfying both calibration and equalized odds is mathematically impossible
CCalibration and equalized odds only conflict when the model has very high accuracy
DThere is no mathematical conflict; disagreement is purely semantic
This is a fundamental impossibility result in fairness: when base rates differ (e.g., disease prevalence differs across demographics, or historical hiring rates differ), satisfying both calibration and equalized odds simultaneously is mathematically impossible. Calibration says P(Y=1|pred=1, group=A) should equal P(Y=1|pred=1, group=B), but equalized odds says FNR should be equal. These constraints are incompatible when base rates differ. Practitioners must choose which fairness definition to prioritize, understanding the implications of this choice for different groups.
Question 4 True / False
What is the difference between individual fairness (similar individuals treated similarly) and group fairness (equitable treatment of demographic groups)?
TTrue
FFalse
Answer: True
Individual fairness asks: do similar individuals (in relevant attributes) receive similar predictions/treatment, regardless of protected attributes? This requires defining similarity metrics on non-protected attributes. Group fairness asks: do demographic groups receive equitable treatment on average? These are complementary: individual fairness prevents arbitrary discrimination but does not address systemic disparities; group fairness addresses systemic disparities but allows individual variation if the group average is fair. A system can satisfy one without the other: it could be individually fair yet have group disparities (if qualifications differ), or group-fair yet individually unfair (if similar individuals are treated differently). Achieving both simultaneously requires careful design and may be impossible in some settings.