Fairness-Aware Evaluation

Slice-based evaluation breaks down model performance across operational segments such as device type, region, query length, or content type. Fairness-aware evaluation applies the same breakdown to protected demographic groups, making fairness a core design requirement rather than a post-launch concern. Consider an ads ranking system that under-delivers impressions to underrepresented or protected demographic groups, a hiring model that disproportionately screens out qualified candidates from protected groups, or a lending model with unexplained approval-rate gaps across protected attributes such as race or gender. These risks are not purely theoretical. Regulatory frameworks and enforcement bodies, including the EEOC, ECOA, and the EU AI Act, increasingly require evidence that high-stakes systems are evaluated for discriminatory impact. System design should account for fairness from the start rather than treating it as a post-launch fix.

This lesson walks through the core fairness metrics and how they compare, introduces counterfactual fairness as a causal lens, covers post-processing calibration techniques applied per demographic group, frames the fairness-accuracy trade-off as a design decision with ethical stakes, and closes with feedback loop bias, the mechanism by which biased models generate increasingly biased data over time.

Core fairness metrics compared

Fairness metrics formalize what “fair” means measurably. The challenge is that different definitions of fairness suit different application contexts, and some definitions are provably incompatible with each other. Four foundational metrics form the vocabulary you need for both production systems and ML system design interviews.

• Demographic parity requires that the positive prediction rate is equal across groups, expressed as $P(\hat{Y}=1 \mid G=a) = P(\hat{Y}=1 \mid G=b)$ ...