The Prosecutor's Fallacy

Bayes' theorem states that the posterior probability of a hypothesis given evidence equals the prior probability of the hypothesis multiplied by the likelihood of the evidence under that hypothesis, divided by the total probability of the evidence. In odds form, the relationship is: posterior odds = prior odds multiplied by the likelihood ratio. The likelihood ratio (LR) is the key forensic quantity: it measures how much more probable the evidence is under the prosecution hypothesis than under the defence hypothesis.

To make this concrete, suppose a DNA profile matches between a crime-scene sample and a suspect. The forensic biologist reports a random match probability (RMP) of 1 in 1,000,000. This means P(match | innocent, randomly chosen person) = 0.000001. The LR is approximately 1,000,000 in favour of the prosecution hypothesis that the suspect is the source. Now suppose the prior odds of guilt, based on independent evidence such as CCTV placing the suspect at the scene, are 10 to 1 in favour of guilt. The posterior odds are 10,000,000 to 1 in favour of guilt. This is legitimate Bayesian reasoning.

The prosecutor's fallacy short-circuits this process. Instead of reporting the LR, the expert or prosecutor states: 'The probability that the defendant is innocent is 1 in 1,000,000.' This treats the RMP as though it were the posterior probability of innocence. But P(innocence | evidence) depends on the prior probability of innocence. If the suspect was chosen from a database of everyone in a city of 2 million people, the prior probability of guilt before any forensic evidence is only 1 in 2,000,000. Multiplying by the LR of 1,000,000 gives posterior odds of only 1 to 2 in favour of guilt. The DNA evidence is strong, but it does not establish guilt: the prior must be considered.

The Sally Clark case in England is the most widely cited example. Clark was convicted in 1999 of murdering her two infant sons. At trial, paediatrician Sir Roy Meadow testified that the probability of two sudden infant deaths in the same affluent non-smoking family was 1 in 73 million. This figure was statistically dubious because it assumed the two deaths were independent, which is not warranted given shared genetic and environmental factors. But the deeper probabilistic error was framing the figure as though it represented the probability that Clark was innocent. The Royal Statistical Society issued a statement after the conviction specifically criticising this reasoning. Clark was acquitted on her second appeal in 2003.

The Adams cases in England (R v Adams, 1996 and 1998) raised the same issue in a DNA context. The Court of Appeal accepted expert evidence in the form of a Bayesian calculation, but expressed serious reservations about whether jurors could properly apply Bayes' theorem and whether it was appropriate to ask them to do so explicitly. The court's discomfort with explicit Bayesian calculation by juries led to a practice in which LR evidence is presented verbally rather than numerically in many UK courts, using standardised verbal equivalents from scales developed by bodies such as ENFSI.

In the United States, the issue arose prominently in People v Collins (1968) in California, where a prosecutor had an expert assign probabilities to individual features of two suspects and multiply them together as though the features were independent. The Supreme Court of California reversed the conviction, finding that the statistical method was fundamentally unsound. More recently, the National Research Council report on forensic science (2009) and the PCAST report on forensic evidence (2016) both identified the prosecutor's fallacy as a recurrent source of flawed testimony across multiple evidence types.

The prosecutor's fallacy overstates the evidential weight of a match. The defence fallacy commits the opposite error: it understates the weight by arguing that because many people in the population could match, the evidence is essentially worthless. Suppose a DNA profile has an RMP of 1 in 10,000 in a city of one million people. The defence argues: 100 people in this city could match, so the evidence tells us nothing about the suspect. This is wrong. The evidence reduces the number of plausible suspects from one million to roughly 100 and shifts the posterior probability of guilt substantially, depending on the prior. The defence fallacy ignores this shift.

Base-rate neglect is a related but distinct cognitive error. It occurs when decision-makers ignore the prior probability of a hypothesis entirely and focus only on the match evidence. It is not specific to the prosecution or defence position. A juror who thinks 'this DNA matches, so he must be guilty' and a juror who thinks 'lots of people could match, so the evidence proves nothing' are both neglecting the prior, just in opposite directions. Proper reasoning requires integrating the LR with an explicit prior, even if that prior is expressed qualitatively.

Reporting forensic match evidence as a likelihood ratio rather than as a source probability or match probability is the primary technical safeguard against the prosecutor's fallacy. The LR answers the question: how much more probable is this evidence if the prosecution hypothesis is true than if the defence hypothesis is true? It makes no claim about prior probability and therefore cannot be misread as a posterior probability of guilt or innocence.

In practice, forensic scientists in England and Wales are now guided by the Forensic Science Regulator's Codes of Practice and Conduct, which require evaluative reporting in LR terms. ENFSI's Guideline for Evaluative Reporting in Forensic Science (2015) sets out the same framework across member states in the European Union and associated countries. The guideline defines the prosecution and defence propositions, requires the expert to state the LR explicitly, and specifies a verbal scale for communicating LR values to courts that prefer qualitative language over numbers.

In India, the Bharatiya Sakshya Adhiniyam 2023 (which replaced the Indian Evidence Act 1872) continues to give courts broad discretion in evaluating expert opinion. There is no statutory requirement for LR-formatted reports, and Indian courts have sometimes received forensic match probabilities without adequate explanation of the Bayesian framework. The position is similar in many other common-law jurisdictions: the LR framework exists in guidance but has not been legislated. In the United States, admissibility of probabilistic genotyping systems such as TrueAllele and STRmix has been contested in multiple jurisdictions, with defence challenges arguing that LR outputs from these systems are not adequately validated.

The ENFSI verbal scale for LR values runs from 'weak support' for LR values between 1 and 10, through 'moderate', 'moderately strong', 'strong', 'very strong', and 'extremely strong' support at LR values above 10,000,000. The verbal label is always anchored to a specific proposition pair stated by the expert. An LR of 1,000,000 means: the evidence is one million times more probable if the prosecution proposition is true than if the defence proposition is true. It does not mean the defendant is one million times more likely to be guilty than innocent.

Judges have several procedural tools available. Pre-trial scrutiny of expert reports, now standard in England and Wales under Criminal Practice Direction V 19A, requires that experts state their proposition pairs and the basis for their LR before trial, so errors can be identified before they reach the jury. In jurisdictions using Daubert or Frye admissibility standards in the US, the court can examine whether the statistical method underlying an LR is validated and whether the expert is qualified to use it.

Jury directions are a second safeguard. Several appellate courts, including the Court of Appeal of England and Wales and the High Court of Australia, have issued guidance on how trial judges should direct juries when forensic probability evidence is presented. These directions typically explain the distinction between match frequency and probability of guilt, warn against treating a very small number as conclusive proof, and remind jurors that probability evidence must be considered alongside all other evidence in the case.

A third safeguard is defence expert access. The fallacy is most dangerous when only the prosecution has a forensic expert and the jury hears only one framing of the statistical evidence. Courts in the UK, the US, and Australia have mechanisms for funding defence expert witnesses in cases where forensic probability evidence is central. The European Court of Human Rights has found violations of Article 6 (right to a fair trial) in cases where defendants lacked effective access to expert assistance to challenge forensic evidence.

A forensic scientist authoring an evaluative report should follow a structured approach to avoid committing or enabling the prosecutor's fallacy. First, define the propositions explicitly: what is the prosecution claiming the evidence means, and what is the most realistic alternative? For DNA evidence the propositions might be: Hp (the suspect is the contributor) versus Hd (an unknown unrelated person is the contributor). The LR is the ratio of the probability of the observed profile under Hp to its probability under Hd.

Second, the report should contain no posterior probability statement. Phrases such as 'it is therefore highly probable that the defendant committed this offence' are prohibited under current ENFSI and UK guidance. The expert reports the LR and the verbal equivalent. Posterior probability is for the trier of fact, not the scientist.

Third, when giving oral evidence, an expert should anticipate the common ways the prosecutor's fallacy is introduced under cross-examination or in closing argument. If a barrister or attorney inverts the conditional and asks the expert to confirm that 'the chance of an innocent person matching is 1 in a million, so the chance of innocence is 1 in a million', the expert must correct the framing on the record. This correction is part of the expert's duty to the court, which overrides the duty to the instructing party.