Foundational Validity as Applied
Definition
PCAST's standard requiring that a forensic discipline be validated through black-box studies measuring real-examiner error rates under realistic conditions, not just laboratory demonstration of the underlying science.
- Source
- PCAST 2016 report
- Requires
- Black-box studies of real-examiner error rates
- Contrasts with
- Laboratory demonstration of underlying science alone
- Applies to
- Pattern and impression evidence disciplines
Common questions
What is a black-box study and why does PCAST insist on it specifically?+
A black-box study has examiners analyse realistic test samples with known ground truth without knowing the correct answer, measuring how often they err in practice. PCAST required this because knowing a method is scientifically sound does not tell you how often trained humans actually make mistakes applying it.
How does foundational validity as applied differ from validity as applied to a specific case?+
Foundational validity establishes whether a method reliably works at all, across studies and examiners generally. Validity as applied asks whether a particular examiner, following the method correctly, reached a reliable conclusion in a particular case, which is a separate, case-specific question PCAST also addressed.
Which forensic disciplines did PCAST find lacked adequate foundational validity studies at the time of the report?+
PCAST found bitemark comparison lacked foundational validity, while firearms/toolmark and latent fingerprint comparison had only a small number of appropriately designed black-box studies, calling for more independent replication before treating error rates as well established.
Related terms
- Courtroom-Language Translation Problem
- The empirically documented gap between what forensic verbal conclusion terms (such as 'strong probability') are intended to convey by examiners and what...
- Elimination (SWGDOC)
- The most definitive negative conclusion on the SWGDOC scale, meaning fundamental differences in writing habits that cannot be attributed to disguise, natural...
- ENFSI Evaluative Standard
- The European Network of Forensic Science Institutes 'Standard for Formulation of Evaluative Forensic Science Expert Opinions' (2015/2016), which requires examiners to reason...
- Identification (SWGDOC)
- The most positive conclusion on the SWGDOC scale, meaning the examiner has found individualising agreement in all significant habitual writing characteristics, with...
- Likelihood Ratio (LR)
- The ratio of two conditional probabilities: the probability of the observed evidence given the prosecution's hypothesis (same source), divided by the probability...
- No Conclusion (SWGDOC Position 5)
- The midpoint of the SWGDOC scale, indicating that the evidence is equally consistent with a common source or different sources. It is...
- PCAST 2016 Report
- The President's Council of Advisors on Science and Technology 2016 report 'Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods,'...
- Population-Frequency Anchoring
- The practice of grounding a conclusion about feature rarity in empirically measured frequency data for those features in the relevant writing population,...
- SWGDOC Nine-Point Scale
- The conclusion scale developed by the Scientific Working Group for Forensic Document Examination, running from 'identification' (most positive) through 'no conclusion' (midpoint)...