Statistical Parity vs. Equal Opportunity: Which Fairness Metric to Use?

Question

Hey everyone! 👋 Trying to wrap my head around fairness metrics in machine learning. Statistical Parity vs. Equal Opportunity...which one should I be using? 🤔 Help!

hamilton.carol47 · Accepted Answer

📚 Statistical Parity vs. Equal Opportunity: A Deep Dive
In the quest for fairness in machine learning, two prominent metrics emerge: Statistical Parity and Equal Opportunity. Understanding their nuances is crucial for building equitable AI systems. Let's break them down.

📊 Defining Statistical Parity
Statistical Parity, also known as demographic parity, aims for equal representation in outcomes across different groups. Essentially, it requires that the proportion of individuals receiving a positive prediction should be the same regardless of their group membership (e.g., race, gender).
Mathematically, Statistical Parity is satisfied if: 
$P(\hat{Y} = 1 | G = g_1) = P(\hat{Y} = 1 | G = g_2)$ for all groups $g_1$ and $g_2$,
where $\hat{Y}$ is the predicted outcome and $G$ represents the protected group.

🌍  It ensures that the model's predictions do not disproportionately favor one group over another.
  ⚖️  It can be a useful starting point for identifying potential bias in a system.
  ⚠️  Achieving statistical parity doesn't guarantee overall fairness, as it ignores whether the positive predictions are actually correct.

🎯 Defining Equal Opportunity
Equal Opportunity focuses on ensuring that individuals who truly deserve a positive outcome (i.e., those with $Y=1$) have an equal chance of receiving a positive prediction, regardless of their group membership.
Mathematically, Equal Opportunity is satisfied if:
$P(\hat{Y} = 1 | Y = 1, G = g_1) = P(\hat{Y} = 1 | Y = 1, G = g_2)$ for all groups $g_1$ and $g_2$,
where $Y$ is the actual outcome.

✅ It addresses the fairness of positive predictions for those who should legitimately receive them.
  🔑 It targets the true positive rate (TPR) and seeks to equalize it across groups.
  🧐 It doesn't consider false positives, meaning that one group could still be unfairly burdened with incorrect positive predictions.

📝 Statistical Parity vs. Equal Opportunity: A Side-by-Side Comparison

Feature
      Statistical Parity
      Equal Opportunity

Focus
      Equal proportion of positive predictions across groups
      Equal true positive rates across groups

Conditional On
      Group membership only
      Group membership and true positive status

Mathematical Expression
      $P(\hat{Y} = 1 | G = g_1) = P(\hat{Y} = 1 | G = g_2)$
      $P(\hat{Y} = 1 | Y = 1, G = g_1) = P(\hat{Y} = 1 | Y = 1, G = g_2)$

Potential Issue
      Ignores whether positive predictions are correct
      Ignores false positives

Use Case
      When overall representation is critical
      When ensuring fair positive outcomes for deserving individuals is paramount

💡 Key Takeaways

🎯 Both Statistical Parity and Equal Opportunity are valuable fairness metrics, but they address different aspects of fairness.
  🧪  The choice between them depends on the specific context and the priorities of the application.
  🧠  In many cases, a combination of fairness metrics and careful consideration of the potential harms of a system are necessary to achieve truly equitable outcomes.

Statistical Parity vs. Equal Opportunity: Which Fairness Metric to Use?

1 Answers

📚 Statistical Parity vs. Equal Opportunity: A Deep Dive

📊 Defining Statistical Parity

🎯 Defining Equal Opportunity

📝 Statistical Parity vs. Equal Opportunity: A Side-by-Side Comparison

💡 Key Takeaways

Join the discussion

Feature	Statistical Parity	Equal Opportunity
Focus	Equal proportion of positive predictions across groups	Equal true positive rates across groups
Conditional On	Group membership only	Group membership and true positive status
Mathematical Expression	$P(\hat{Y} = 1 \| G = g_1) = P(\hat{Y} = 1 \| G = g_2)$	$P(\hat{Y} = 1 \| Y = 1, G = g_1) = P(\hat{Y} = 1 \| Y = 1, G = g_2)$
Potential Issue	Ignores whether positive predictions are correct	Ignores false positives
Use Case	When overall representation is critical	When ensuring fair positive outcomes for deserving individuals is paramount