Expert Testimony and the 2009 NAS Critique of Fingerprint ID

Chapter 5 of the NAS report, titled "Friction Ridge Analysis," identified three structural deficiencies in the scientific foundation of fingerprint individualization as it was being practised and testified to in courts.

First, the individuality assumption. Fingerprint examiner testimony routinely asserted that no two fingers produce identical friction-ridge patterns, even in identical twins. The committee found this assumption untested. No population study had measured the probability of two different fingers sharing a set of friction-ridge characteristics of the type typically used in identification conclusions. The claim of uniqueness was a professional assertion, not a demonstrated empirical fact. The committee distinguished between the plausibility of the uniqueness hypothesis (which may well be true) and its demonstration (which had not occurred). Evidence law in all major jurisdictions requires that scientific claims be grounded in demonstrated reliability, not professional tradition.

Second, the transferability problem. Even if uniqueness were assumed, the process of transferring an impression from a finger to a surface and recovering it through development introduces distortion, partial contact, substrate interference, and degradation. Two lifts of the same finger under different conditions may look substantially different. The committee noted that no validated method existed for quantifying how much distortion was acceptable before an identification conclusion became unreliable.

Third, the zero-error-rate claim. Fingerprint examiners in US courts in the early 2000s regularly testified that ACE-V had a zero error rate, or that no verified erroneous identification had ever been made. The committee noted that the Mayfield misidentification of 2004 (a verified erroneous identification confirmed by the FBI's own post-incident review) made this claim factually incorrect as of 2004. More importantly, the committee noted that error-rate estimates require structured empirical studies, not anecdotal casework records, and that no such studies had been conducted.

The committee's recommendations for fingerprint analysis included: conducting population studies to quantify the probability of particular feature sets appearing by chance in unrelated fingers; conducting studies to quantify the error rates of experienced examiners under operational conditions; developing validated statistical models for fingerprint conclusions that could be expressed in probabilistic terms; and requiring that testimony be modified to reflect the genuine uncertainty in the individualization claim.

The most significant empirical response to the NAS report's error-rate recommendation was Ulery, B.T., Hicklin, R.A., Buscaglia, J. and Roberts, M.A. (2011) "Accuracy and reliability of forensic latent fingerprint decisions," published in the Proceedings of the National Academy of Sciences 108(19): 7733-7738.

The study, conducted at the FBI Laboratory and at several cooperating agencies, presented 169 latent print examiners with 744 pairs of latent prints and exemplars, drawn from operational casework to represent the realistic difficulty range that examiners encounter. The examiners did not know which pairs were from the same finger (mates) and which were from different fingers (non-mates). They were asked to reach a conclusion (identification, inconclusive, or exclusion) for each pair.

The key findings were as follows. The false-positive rate (an identification conclusion reached for a non-mate pair) was 0.1 per cent: examiners made identification errors on approximately 1 in 1,000 non-mate comparisons. The false-negative rate (a failure to identify a mate pair) was higher, at approximately 7.5 per cent, primarily because examiners more often reached inconclusive conclusions on difficult pairs rather than erroneous exclusions. These numbers had immediate implications for expert testimony.

First, 0.1 per cent is not zero. The zero-error-rate testimony that had been standard practice before 2009 was empirically refuted. The Ulery false-positive rate gave courts, for the first time, a documented empirical estimate of the risk of an erroneous fingerprint identification. Second, the false-negative rate of 7.5 per cent revealed that the primary conservatism of experienced examiners is to not reach identification conclusions on difficult marks, not to make erroneous exclusions. This pattern is consistent with a professional norm that treats a missed identification as less professionally consequential than a false identification.

The Pacheco, M.L. et al. (2014) follow-on study "Miami-Dade Research Study for the Reliability of the ACE-V Process: Accuracy & Precision in Latent Fingerprint Examinations" conducted at the Miami-Dade Police Department Laboratory replicated and extended the Ulery findings using a different examiner population and different mark sets, finding similar false-positive rates and providing geographic generalizability to the FBI study findings.

Before the NAS report, standard fingerprint identification testimony in US courts followed a categorical pattern: the examiner identified the latent mark as having been made by the named individual to the exclusion of all other persons in the world. This language asserted both a positive identification and a global exclusion. Both assertions were, in the committee's view, scientifically unsupportable as categorical statements.

The SWGFAST response, in its 2013 "Standards for Conclusions" document, recommended a graduated conclusion scale: "identification" (formerly standard), "probable identification," "inconclusive," "probable exclusion," and "exclusion." The introduction of "probable identification" as an intermediate category acknowledged that some cases do not have sufficient friction-ridge detail to support a categorical identification but do have sufficient detail to support a lower-confidence conclusion. SWGFAST did not, however, recommend a move to full probabilistic (LR-based) reporting, which remains contested in US operational practice.

The OSAC Friction Ridge Subcommittee's Standard for Friction Ridge Examination Conclusions (OSAC 2022/0004, published 2023) adopted a similar graduated scale. The standard specifies the documentation requirements for each conclusion level and explicitly prohibits the use of "zero error rate" or "certain" language in examination reports and testimony. The standard does not require probabilistic (LR-based) reporting but does require that testimony be consistent with the empirical error-rate literature, which means acknowledging the 0.1 per cent false-positive rate context when asked about error rates.

The ENFSI evaluative-reporting framework, set out in the ENFSI Guideline for Evaluative Reporting in Forensic Science (2015) and applied to fingerprint examination in the ENFSI FWG guidance, goes further than OSAC and SWGFAST. The ENFSI position is that conclusions should be expressed as likelihood ratios: the probability of observing the friction-ridge evidence if the mark came from the suspect, divided by the probability of observing the same evidence if it came from an unknown different person. This requires a statistical model of friction-ridge feature probability, which several European research groups (including the Netherlands Forensic Institute and the Lausanne School of Criminal Sciences) have been developing. Full LR reporting for fingerprint evidence is operationally used in the Netherlands, Switzerland, and to a limited extent in the UK, but is not yet the majority practice in most European institutes.

In India, the Bharatiya Sakshya Adhiniyam 2023 (BSA 2023, replacing the Indian Evidence Act 1872) governs the admissibility of expert opinions in court proceedings. Under section 39 BSA 2023 (the provision governing opinions of experts), courts admit fingerprint expert evidence on essentially the same basis as before, with no specific language requirement analogous to the OSAC or ENFSI standards. However, the post-NAS literature is increasingly cited by Indian defence counsel in challenging categorical fingerprint testimony, and several High Court judgments have noted the absence of error-rate disclosure in expert reports as a factor relevant to the weight of the evidence.

The Daubert standard (Daubert v. Merrell Dow Pharmaceuticals, US Supreme Court, 1993) requires federal trial judges in the United States to assess whether proposed expert testimony is based on a methodology that is sufficiently reliable to assist the trier of fact. The Daubert inquiry considers whether the methodology has been tested, whether it has been subject to peer review and publication, whether the known or potential error rate is acceptable, and whether the methodology is generally accepted in the relevant scientific community. Federal Rule of Evidence 702 (as amended in 2011 and again in 2023) operationalises this inquiry.

Fingerprint identification has survived Daubert challenges in all major US federal circuit courts that have considered the question, including United States v. Llera Plaza (3d Cir. 2002), United States v. Baines (10th Cir. 2009), and United States v. Rose (4th Cir. 2011). In each case, courts upheld admissibility while acknowledging the scientific limitations identified in the NAS report. Post-Ulery, the evidentiary landscape has shifted: courts now have empirical error-rate data available, and expert witnesses are expected to acknowledge the Ulery findings when asked about error rates under cross-examination. The 2023 amendment to FRE 702 tightened the sufficiency requirement, specifying that an expert must demonstrate, not merely assert, that the proposed opinion is based on sufficient facts or data.

In England and Wales, the Criminal Practice Directions and CrimPR Part 19 govern expert witness disclosure obligations. The Criminal Procedure Rules require fingerprint expert witnesses to disclose their methodology, its limitations, and any material weaknesses in their conclusion. Post-NAS, this disclosure obligation effectively requires expert witnesses to acknowledge the Ulery error-rate data when providing fingerprint identification evidence. The BSA 2023 amendment to the Forensic Science Regulator Act 2021 reinforced this by requiring providers to comply with FSR Codes, which include the evaluative-reporting guidance.

The Indian context under BSA 2023 is governed by the general expert witness provisions rather than by discipline-specific admissibility rules. Unlike the Daubert framework, the Indian courts do not apply a gatekeeping test requiring pre-trial demonstration of reliability. Expert evidence is admitted more permissively, with reliability arguments going to weight rather than admissibility. However, Indian High Courts and the Supreme Court have increasingly engaged with the reliability literature. The Supreme Court's judgment in State of Maharashtra v. Dr. Praful B. Desai (2003) and the Delhi High Court's treatment of digital forensic evidence in several post-2015 cases suggest that courts are willing to engage with methodological reliability arguments when they are clearly put. Defence counsel who have deployed the NAS report and the Ulery data in cross-examination of fingerprint experts in Indian courts have reported judicial interest in the error-rate findings.

In Australia, the Evidence Act framework varies by jurisdiction (Commonwealth, NSW, Victoria, Queensland, etc.) but follows a broadly common law approach. The Australian and New Zealand Forensic Science Society (ANZFSS) has produced guidance that reflects the post-NAS consensus on fingerprint testimony, and the Australian courts have in several post-2012 cases received evidence about the NAS critique and the Ulery findings. The NSW Court of Criminal Appeal's treatment of fingerprint evidence in R v. Pahuja (2017) addressed the admissibility of probabilistic reporting frameworks for fingerprints, acknowledging that the field was in transition.

One practical response to the post-NAS credibility questions around fingerprint testimony has been the development of structured checklists that courts, counsel, and expert reviewers can use to assess whether a specific piece of fingerprint evidence meets a minimum reliability threshold. The most widely cited in the UK context is the reliability framework derived from the Law Commission's 2011 report on expert evidence and formalised in the Cairns decision (R v. Cairns and others, Court of Appeal England and Wales, 2013), which specified criteria for assessing whether an expert's methodology and conclusion were sufficiently reliable to be placed before a jury.

For fingerprint evidence specifically, the reliability indicators that courts and reviewing parties now examine include: whether the examination was conducted by an accredited laboratory; whether the examiner holds a recognised competency credential; whether the examination was subject to independent verification (and whether that verification was blind or non-blind); whether the examiner documented the features used in reaching the conclusion before the verification step; whether the mark quality was assessed against a documented quality-assessment scale; whether the contextual information management protocol was applied; and whether the error rates documented in the Ulery study and its successors were considered and disclosed.

In the US context, the Starzecpyzel framework (United States v. Starzecpyzel, SDNY 1995) and the subsequent development of Daubert fingerprint case law have produced a similar reliability checklist, though the US framework gives more weight to general acceptance in the scientific community (one of the Daubert factors) and less weight to the specific quality controls of the individual case examination.

The ENFSI position, aligned with the evaluative reporting framework, effectively replaces the binary admissibility/exclusion question with a continuous weight-of-evidence question. If a fingerprint opinion is expressed as a likelihood ratio with a documented statistical model, the court is provided with a directly interpretable probabilistic statement about the strength of the evidence. If the opinion is expressed categorically, the court must rely on the reliability indicators listed above to assess the weight it deserves. The direction of travel across the major common-law jurisdictions is clearly toward the structured reliability checklist approach, even if full probabilistic reporting is not yet universal.

SWGFAST's 2013 "Standards for Conclusions" was a turning-point document. It introduced the graduated conclusion scale and, critically, specified that expert witnesses in US criminal proceedings should not use language that implies certainty beyond what the empirical literature supports. The phrase "to the exclusion of all others" was flagged as problematic: it claims a global negative (no other person in the world made this mark) that cannot be tested and that the population studies called for by the NAS report had not established.

The OSAC 2022/0004 standard (published 2023) went further in specifying the vocabulary. It defined "identification" as a conclusion that the questioned friction ridge impression was made by the person associated with the known exemplar, with a recognition that the probability of an erroneous conclusion is greater than zero. This definition acknowledges that "identification" is a probabilistic conclusion above a threshold, not a categorical statement of zero error probability. The standard explicitly prohibited testimony claiming zero error rates and required that examiners be able to characterize their conclusions in terms consistent with the Ulery-type empirical data when asked under cross-examination.

The UK FSR fingerprint guidance aligns with this shift. The Fingerprint Source Book and the FSR Codes require that conclusions be reported in language that accurately represents the strength of the evidence and does not overstate the certainty of the finding. The FSR has not mandated full LR-based reporting across all UK fingerprint bureaux, but it has endorsed a directional movement toward evaluative reporting and has published supporting guidance for laboratories that wish to adopt probabilistic frameworks.

The ENFSI LR framework applied to friction-ridge examination uses population data from large reference databases to estimate the denominator of the likelihood ratio: the probability that a randomly selected finger would display the particular combination of friction-ridge features observed in the latent mark. The Netherlands Forensic Institute (NFI) has been a leader in developing and applying this framework, using AFIS database statistics and morphological feature distributions from the Dutch population register fingerprint collection. The Swiss Federal Institute of Police Science (fedpol) applies a similar approach in the Swiss criminal jurisdiction.

1. Examine and document mark quality
   Apply the laboratory's documented quality scale (e.g. the Ashbaugh sufficiency scale or OSAC quality tiers) to assess whether the mark has sufficient detail for a meaningful comparison. Record quality assessment before comparison.
2. Conduct and document ACE-V comparison
   Follow linear ACE-V with sequential unmasking (see Module 10 companion topic on Dror 2006). Record analysis of the latent mark before introducing the exemplar.
3. Record conclusion at appropriate vocabulary level
   Apply the SWGFAST / OSAC graduated scale. If insufficient detail for a categorical identification, record 'inconclusive' rather than overstating the conclusion. Avoid 'to the exclusion of all others' language.
4. Obtain independent blind verification
   Send the case to a verifying examiner who does not know the primary conclusion. The verification conclusion is recorded independently.
5. Prepare a compliant expert report
   Include the methodology description, the quality assessment, the conclusion and its basis, the verification outcome, and any material limitations or uncertainties, per CrimPR Part 19 (UK) or FRE 702 (US) or the applicable local standard.
6. Prepare for cross-examination on error rates
   Know the Ulery 2011 findings (0.1% false-positive rate, 7.5% false-negative rate). Be able to explain how the quality controls in the specific case (blind verification, documentation, proficiency testing participation) bear on the reliability of the specific conclusion.