Document AI system responsibilities, assess fairness-related harms, and compare mitigation approaches. Use group-level evaluation and stakeholder review while k
Last reviewed: 2026-10-03
Governance connects a system's intended use with review responsibilities and evidence. Fairness assessment is one part of that work: identify potential harms and affected groups, then examine relevant errors and outcomes across groups. Fairlearn provides disaggregated metrics and tools for examining trade-offs.
Bias auditing and mitigation require context. A hiring or public-service exercise should document representation, label quality, sample sizes, and which harms its metrics miss. Comparing mitigation approaches can reveal trade-offs, but no single metric certifies fairness or legal compliance.
Both can be relevant. Statistical analysis measures outcomes, while domain and policy review help determine which harms and obligations matter. State the boundaries of your own assessment.
No. Metrics cover selected aspects of a system, and improvements can introduce other trade-offs. Document remaining concerns and include affected stakeholders in review.