How Numbers Enter Forensic Conclusions

For most of the twentieth century, forensic examiners concluded reports with statements such as 'the questioned and known samples are from the same source' or 'the marks are consistent with having been made by this tool'. These conclusions are categorical: they assert an outcome without quantifying the uncertainty around it. Scientifically, a conclusion without an error rate is not falsifiable in any useful sense. Legally, a jury told only that two things match has no basis for weighing that evidence against other case facts.

The structural problem is that any forensic comparison can be wrong in two directions. A false positive (declaring a match when the items came from different sources) can wrongly implicate an innocent person. A false negative (declaring no match when the items share a source) can let a guilty person go free. Without data on how often each type of error occurs, neither the examiner, the court, nor a reviewing scientist can evaluate the significance of any particular conclusion.

The legal response varied by jurisdiction but converged on similar demands. In the United States, Daubert v. Merrell Dow Pharmaceuticals (1993) placed courts in the role of gatekeepers, requiring that expert testimony be grounded in tested methods with known error rates. In England and Wales, the Criminal Procedure Rules and the Law Commission's 2011 report on expert evidence pressed for greater transparency about the basis and limits of expert opinion. In India, courts applying the Bharatiya Sakshya Adhiniyam 2023 (which replaced the Indian Evidence Act 1872) still evaluate expert evidence through judicial discretion, but the same underlying logic applies: a conclusion without a stated basis is harder to assess. The EU's forensic science standards, developed through the European Network of Forensic Science Institutes (ENFSI), similarly moved toward evaluative reporting as a formal requirement.

The likelihood ratio (LR) is the central tool of modern evaluative forensic reporting. It answers a specific question: given the evidence that was observed, how much more probable is that evidence under the prosecution hypothesis than under the defence hypothesis? Formally, LR = P(evidence | Hp) divided by P(evidence | Hd). An LR of 1000 means the evidence is 1000 times more probable if Hp is true than if Hd is true. An LR of 0.5 means the evidence is twice as probable under Hd, and therefore favours the defence.

The LR is calculated with reference to a specific pair of propositions. Changing the defence proposition changes the LR. If Hd is 'the trace came from the accused's brother' rather than 'the trace came from an unknown unrelated person', the denominator of the LR will be different because brothers share more genetic markers than unrelated individuals. This proposition-dependence is not a weakness of the framework; it forces the examiner to be explicit about what comparison is being made, which is exactly what the court needs.

In DNA analysis, the LR is often computed from a population database: the numerator is the probability of the observed DNA profile given that the accused contributed the sample (usually 1, if the profile is a full match), and the denominator is the random match probability from the relevant reference population. In disciplines without population frequency databases, such as some forms of pattern evidence, the LR may be estimated through empirical studies of how often examiners confuse different sources, or it may be expressed as a bounded range rather than a point estimate.

The LR is not, by itself, the probability that the accused is the source of the trace. To reach that conclusion, a fact-finder must combine the LR with prior information about the probability of the competing hypotheses before the forensic evidence was considered. This is the role of Bayes' theorem. In odds form, the rule is: posterior odds = prior odds multiplied by the LR. The forensic examiner provides the LR; the prior odds are a matter for the court, informed by other case evidence.

Consider a simple illustration. Suppose the prior odds that the accused is the source (based on non-forensic case evidence such as witness testimony and opportunity) are 1:10, meaning 10 times as probable that someone else is the source. A DNA LR of 1,000,000 shifts those odds dramatically: posterior odds = (1/10) multiplied by 1,000,000 = 100,000:1 in favour of the accused being the source. The DNA evidence would be powerful even against an initially sceptical prior. Now suppose the prior odds are 1:1,000,000 (the accused was identified solely by a database trawl with no other evidence linking them to the scene). The same LR now produces posterior odds of only 1:1. The evidence is still consistent with guilt, but it is not, on its own, proof.

Bayesian reasoning is not limited to DNA. Voice comparison laboratories in Europe and Australia now publish LR-based reports. Footwear mark examiners, digital forensics practitioners, and document examiners in jurisdictions with active forensic standards programmes, including the Netherlands Forensic Institute and the UK Forensic Science Service successor organisations, have moved toward LR-framed conclusions. The statistical machinery differs by discipline, but the logical structure is the same.

The introduction of numerical evidence into courtrooms also introduced a set of predictable reasoning errors. These errors are well documented in the psychology and legal literature. Forensic practitioners need to recognise them both in their own reasoning and in the questions they are asked in cross-examination.

• Prosecutor's fallacy: confusing P(evidence | innocent) with P(innocent | evidence). If the random match probability is 1 in a million, that is P(evidence | innocent person is source). It is not P(accused is innocent | evidence). The two are related by Bayes' theorem and can differ enormously depending on the prior odds.
• Defence attorney's fallacy: arguing that because 1 in a million people share the profile and there are millions of people in the country, there must be many potential alternative sources, so the evidence proves nothing. This ignores the prior probability. If the accused was already identified by non-forensic evidence, the relevant population is not the whole country.
• Transposition of the conditional: a general form of both fallacies above. Treating P(A | B) as if it equals P(B | A). In probabilistic testimony, this occurs whenever an examiner states the probability of the evidence as if it were the probability of the hypothesis.
• The uniqueness assumption: claiming that because a feature appears unique, identification is certain. Uniqueness in a sample does not prove uniqueness in the world. Without a population study showing how rare the feature is, uniqueness is an assertion, not a measurement.
• Confirmation bias in LR estimation: examiners who know the case context before conducting their analysis are at risk of unconsciously adjusting their findings toward the expected result. Blind verification protocols, in which a second examiner reviews the conclusion without knowledge of the first examiner's opinion, are a standard mitigation.

The R v Adams cases in England (1996, 1998) are among the most studied examples of Bayesian reasoning entering court proceedings and producing confusion. The Court of Appeal ultimately discouraged presenting Bayes' theorem directly to juries, preferring that jurors use their intuitive reasoning rather than a formal calculation. This does not mean the reasoning is wrong; it means courts prefer that experts absorb the calculation and present a verbal conclusion calibrated to the LR value, rather than asking jurors to do arithmetic.

Evaluative reporting translates the likelihood ratio into a structured expert report that courts can use. The ENFSI Guideline for Evaluative Reporting in Forensic Science (2015, updated) sets out the framework most widely adopted in Europe and influential in Commonwealth jurisdictions. It requires the examiner to state the propositions being compared, describe the findings, state the probability of the findings under each proposition, and express the LR or its verbal equivalent. The report does not state the posterior probability of guilt; that is the court's function.

Verbal equivalence scales translate an LR value into a phrase the court can use without being asked to process a number. The ENFSI scale runs from 'weak support' (LR slightly above 1) through 'moderate', 'moderately strong', 'strong', and 'very strong' to 'extremely strong support' (LR above 10,000,000). The UK Forensic Science Regulator's codes of practice require accredited laboratories to use a defined verbal scale and to be able to justify the mapping of their LR estimate to the chosen verbal category. Australia's National Institute of Forensic Science has published similar guidance.

Courts differ in how readily they accept evaluative reports. In England and Wales, the LR framework is well established in DNA evidence and has been applied in voice comparison, footwear, and some pattern disciplines. In the United States, the National Commission on Forensic Science (active 2013 to 2017) and the Organisation of Scientific Area Committees (OSAC) have pushed for validated methods with defined error rates, though the LR is less uniformly adopted across disciplines than in the UK. In India, the Bharatiya Sakshya Adhiniyam 2023 gives courts broad latitude in evaluating expert opinion, and the forensic science infrastructure is still developing the laboratory accreditation systems needed to produce evaluative reports consistently.

The LR framework is logically sound, but it depends on inputs that must themselves be validated. A DNA LR rests on allele frequencies derived from population databases; those databases must be representative of the population that is actually relevant to the case. A footwear mark LR rests on studies of how often marks from different shoe types are confused by examiners; those studies must be designed to reflect real casework conditions. If the inputs are wrong, the LR is wrong regardless of how correctly the formula is applied.

Validation studies measure two things: sensitivity (the ability to detect a true correspondence) and specificity (the ability to avoid false correspondences). In black-box studies, examiners are given sets of trace samples and reference samples, some from the same source and some not, without knowing which is which, and their decisions are recorded. The false positive rate and false negative rate from these studies are direct empirical estimates of the error rates that underlie the LR framework. The FBI's black-box studies on firearms and toolmarks, and the fingerprint proficiency studies run by NIST, are examples of this approach.

The framework also has limits in disciplines where population databases do not exist or are too small to support reliable frequency estimates. For handwriting, some questioned-document features, and complex mixed DNA profiles with limited data, examiners may need to express the LR as a range or acknowledge that only a qualitative assessment is currently possible. Honesty about these limits is itself a form of evaluative reporting; overstating precision is as misleading as understating evidence strength.