EU AI Act Art 10 (Training Data Requirements)
Definition
Provision requiring that training, validation, and testing datasets for high-risk AI systems be relevant, representative, free of errors, and complete, with appropriate measures to identify and address possible biases. For biometric identification systems, this requires demographic representativeness and disclosed demographic-effects test results.
- Provision
- Article 10, EU AI Act
- Applies to
- High-risk AI systems, including biometric identification
- Core requirement
- Relevant, representative, error-free, complete datasets
- Biometric-specific requirement
- Demographic representativeness and disclosed bias testing
Common questions
Why does Article 10 specifically require demographic representativeness for biometric systems?+
Face and fingerprint matching algorithms trained on unrepresentative datasets have documented accuracy disparities across demographic groups, so the provision requires training data spread across relevant demographics to reduce the risk of systematically worse performance for some groups.
What must a provider disclose under the demographic-effects testing requirement?+
Providers must document and disclose the results of testing that measures whether the system's accuracy differs across demographic groups, giving regulators and downstream users visibility into bias the model may carry before it is deployed.
Does Article 10 apply to biometric systems built before the AI Act took effect?+
High-risk systems already on the market generally face phased compliance deadlines rather than an immediate retroactive requirement, though providers must bring qualifying systems into compliance within the timelines the Act sets for existing high-risk AI.
Related terms
- AFIS (Automated Fingerprint Identification System)
- An automated system that compares a fingerprint query (typically a latent print from a crime scene) against a reference database to identify...
- Clearview AI
- New York company that built a face-recognition database of approximately 30 billion images scraped from public websites without consent; subject to BIPA...
- Daubert Standard
- The US federal evidentiary standard (Daubert v. Merrell Dow Pharmaceuticals, 1993) requiring that expert testimony be based on scientifically valid methods with...
- Disparate Impact
- A legal and statistical concept describing a policy or practice that has a disproportionate adverse effect on a protected group, regardless of...
- False-Positive Rate (FMR)
- In biometric verification: the probability that a comparison of images of two different individuals is incorrectly declared a match. In forensic identification,...
- Fitzpatrick Scale
- A skin-tone classification system originally developed by dermatologist Thomas B. Fitzpatrick in 1975 (initially covering four types for UV response in phototherapy),...
- Gender Shades (2018)
- Study by Joy Buolamwini and Timnit Gebru published at FAccT 2018 that measured accuracy disparities in three commercial face-recognition APIs across intersectional...
- NIST FRVT (Face Recognition Vendor Test)
- Ongoing NIST evaluation of commercial face-recognition algorithms, with the 2019 Demographic Effects report covering 189 algorithms from 99 developers; confirms Gender Shades...
- Robert Williams Case (2020)
- First publicly documented wrongful arrest in the US attributed to a face-recognition false positive: Detroit police arrested Robert Williams, a Black man,...