Gender Shades (2018)
Definition
Study by Joy Buolamwini and Timnit Gebru published at FAccT 2018 that measured accuracy disparities in three commercial face-recognition APIs across intersectional gender-skin-tone categories; the first controlled published evidence of systematic disparity, with darker-skinned female faces showing up to 35 percentage-point higher error rates than lighter-skinned male faces.
- Authors
- Joy Buolamwini and Timnit Gebru
- Venue
- FAccT 2018
- Method
- Measured accuracy across intersectional gender-skin-tone categories
- Systems tested
- Three commercial face-recognition APIs
- Key finding
- Up to 35 percentage-point higher error for darker-skinned women vs lighter-skinned men
Common questions
Why did the study test intersectional categories rather than gender or skin tone separately?+
Testing gender and skin tone in isolation can mask a disparity that only appears in combination, and the study found darker-skinned women were misclassified far more often than either darker-skinned men or lighter-skinned women alone would predict, showing the intersectional grouping was necessary to reveal the true gap.
What impact did this study have beyond the research literature?+
It prompted several of the tested vendors to publicly commit to bias remediation, influenced subsequent NIST facial recognition accuracy testing to report demographic breakdowns, and became a frequently cited reference in policy debates over regulating law-enforcement use of face recognition.
Does the finding mean all face-recognition systems have the same bias pattern?+
No, accuracy and its demographic distribution vary by vendor, training data, and algorithm version, so the specific error rates reported for the three systems tested in 2018 do not automatically generalise to other systems or to later versions of the same systems.
Related terms
- AFIS (Automated Fingerprint Identification System)
- An automated system that compares a fingerprint query (typically a latent print from a crime scene) against a reference database to identify...
- Clearview AI
- New York company that built a face-recognition database of approximately 30 billion images scraped from public websites without consent; subject to BIPA...
- Daubert Standard
- The US federal evidentiary standard (Daubert v. Merrell Dow Pharmaceuticals, 1993) requiring that expert testimony be based on scientifically valid methods with...
- Disparate Impact
- A legal and statistical concept describing a policy or practice that has a disproportionate adverse effect on a protected group, regardless of...
- EU AI Act Art 10 (Training Data Requirements)
- Provision requiring that training, validation, and testing datasets for high-risk AI systems be relevant, representative, free of errors, and complete, with appropriate...
- False-Positive Rate (FMR)
- In biometric verification: the probability that a comparison of images of two different individuals is incorrectly declared a match. In forensic identification,...
- Fitzpatrick Scale
- A skin-tone classification system originally developed by dermatologist Thomas B. Fitzpatrick in 1975 (initially covering four types for UV response in phototherapy),...
- NIST FRVT (Face Recognition Vendor Test)
- Ongoing NIST evaluation of commercial face-recognition algorithms, with the 2019 Demographic Effects report covering 189 algorithms from 99 developers; confirms Gender Shades...
- Robert Williams Case (2020)
- First publicly documented wrongful arrest in the US attributed to a face-recognition false positive: Detroit police arrested Robert Williams, a Black man,...