Data Leakage
Definition
Use of information from evaluation samples during model development that can overstate performance. Information from test specimens can enter model development through training, tuning or related samples. This can make evaluation performance look better than genuinely independent use.
- Context
- Model evaluation
For example, photographs of the same underlying specimen placed in both training and test sets can overstate performance on genuinely new material.
Keep evaluation material appropriately separated from development. Random splitting does not remove leakage when close relatives or repeated observations occur in both sets.
Related terms
- Alternative-Location Sample
- Material collected to address a relevant possible source other than the proposed scene. A reference should represent a relevant proposed environment and...
- Evaluative Reporting
- A reporting framework that assesses the strength of findings under competing propositions, often using likelihood ratios or a verbal scale. The ENFSI...
- Procedural Control
- Material or a blank used to assess contamination or process behaviour. Controls examine relevant background from devices, reagents or processing. Their selection...