The Defence Fallacy

The defence fallacy appears in a recognisable form. A forensic scientist testifies that the probability of a random person matching the DNA profile found at the crime scene is one in a million. Defence counsel responds: a country of 60 million people would therefore contain approximately 60 people who match the profile. The suspect is just one of 60 innocent matches. The evidence therefore identifies no one.

The arithmetic is correct. In a population of 60 million, a profile present in one person in a million is expected in about 60 people. The logical error is what follows: the claim that this arithmetic renders the DNA evidence valueless ignores every other piece of evidence in the case. The probability that the suspect is guilty is not derived from the match probability alone. It is derived from the match probability combined with the prior probability of guilt, which already incorporates the location of the crime, the suspect's proximity, any witness evidence, CCTV footage, alibis, and all other facts established by the investigation.

Suppose that before the DNA evidence is introduced, the other evidence in the case establishes a prior probability of guilt of 0.5 for the suspect. The DNA match with an RMP of one in a million then updates that probability. The posterior probability of guilt, using Bayes' theorem, is approximately 0.999999. The existence of 59 other people in the country who also match the profile does not change this calculation, because those other 59 people had a near-zero prior probability of being at the crime scene.

Both the prosecutor's fallacy and the defence fallacy are errors in handling conditional probability. The precise statement of the two errors is:

The key insight is that P(E | H0), the probability of observing the DNA match given that an innocent random person left the sample, is not the same as P(H0 | E), the probability that the suspect is innocent given the observed match. Bayes' theorem governs the relationship between these quantities, and it requires knowledge of the prior probability. The defence fallacy error is to use P(E | H0) as if it were the only relevant calculation, and to derive P(H0 | E) directly from population arithmetic.

See Role of Statistics in Evidence Evaluation for a fuller treatment of how the likelihood ratio framework is applied in evaluative reporting.

The defence fallacy typically invokes the entire national population as the pool of potential matches. This is rarely the correct population for evaluating a forensic match. The relevant population is the set of people who could plausibly have left the sample, given all the other evidence. A burglary that occurred in a specific neighbourhood in Mumbai, committed between 2 a.m. and 4 a.m. on a Tuesday, is not plausibly committed by a random draw from India's 1.4 billion people. The relevant population is far smaller: people known to be in that area at that time, people without alibi, people with prior connection to the premises.

The same logic applies in the United States, the United Kingdom, Australia, or any other jurisdiction. A crime scene DNA profile matched against a national database returns a random match probability relative to the database population. But if the investigation has already placed the suspect at the scene through other evidence, the effective prior probability of guilt is much higher than 1/N where N is the national population. Using the national population in the denominator of a defence argument is a rhetorical move, not a statistical one.

The Forensic Science Regulator in England and Wales, in guidance issued from 2012 onward, requires forensic scientists to avoid presenting match probabilities in a way that invites either the prosecutor's fallacy or the defence fallacy. The guidance recommends expressing DNA evidence as a likelihood ratio and being explicit about the population assumptions underlying the calculation. Similar recommendations appear in guidance from the European Network of Forensic Science Institutes (ENFSI) and the Scientific Working Group for DNA Analysis Methods (SWGDAM) in the United States.

Bayes' theorem in the odds form is the clearest tool for showing why the defence fallacy fails. The posterior odds of guilt equal the prior odds multiplied by the likelihood ratio. Written as an equation: Posterior odds = Prior odds x LR, where LR = P(E | Hp) / P(E | Hd), Hp is the prosecution hypothesis (suspect left the sample), and Hd is the defence hypothesis (a random person left the sample).

If the RMP is one in a million, then P(E | Hd) = 0.000001. If the suspect left the sample, the evidence is certain, so P(E | Hp) = 1. The LR is therefore 1,000,000. Now suppose the prior odds of guilt, derived from non-DNA evidence, are 1:1 (an even chance the suspect is the perpetrator). The posterior odds are 1,000,000:1, corresponding to a posterior probability of guilt of approximately 0.9999990. Adding 59 other people in the national population who also match the profile does not change this calculation. Those 59 people each had near-zero prior odds; their posterior odds after multiplying by the same LR of 1,000,000 are still negligible.

The defence fallacy effectively sets the prior odds at 1/(national population), which would be correct only if the suspect were identified by a cold database trawl from the entire population with no other evidence. In any case where non-DNA evidence already implicates the suspect, this prior is wrong, and the fallacy follows from using it.

For a deeper treatment of how prior probabilities are constructed from non-scientific evidence, see Numbers in Forensic Conclusions.

The defence fallacy has been addressed directly in a number of reported cases. In England and Wales, R v Adams [1996] EWCA Crim 222 saw the Court of Appeal grapple with the use of Bayes' theorem by the defence expert. The court recognised both the prosecutor's fallacy and the defence fallacy as errors but expressed discomfort with directing juries to perform explicit Bayesian calculations. Subsequent cases, including R v Deen (1994) and R v Doheny and Adams [1997] EWCA Crim 222, refined the rules on how DNA match statistics may be presented.

In the United States, People v Collins (1968) 68 Cal 2d 319 is an early case addressing misuse of probability in court, where the court rejected a probabilistic argument partly because the underlying population assumptions were unverified. More recent DNA cases have addressed the population denominator problem. The National Academy of Sciences report Strengthening Forensic Science in the United States (2009) and the President's Council of Advisors on Science and Technology (PCAST) report (2016) both identify flawed statistical reasoning, including the defence fallacy structure, as a problem in forensic evidence presentation.

In India, the Bharatiya Sakshya Adhiniyam 2023 (which replaced the Indian Evidence Act 1872) governs the admissibility of expert evidence. Section 39 of the BSA 2023 (equivalent to former Section 45) allows courts to receive expert opinions but leaves weight to the court. Indian courts have not yet developed a body of case law as detailed as the English or US cases on forensic statistics, but the same underlying principles apply: match probability evidence must be presented with its correct context or it misleads the tribunal.

Australian courts addressed the defence fallacy structure in R v Karger [2002] SASC 294, where the Supreme Court of South Australia examined the proper way to present DNA evidence and considered the argument that population size dilutes the evidential value of a match. The court rejected the fallacy in that form. In the European Union, the ENFSI Guideline for Evaluative Reporting in Forensic Science (2015) requires likelihood ratio presentation for precisely this reason.

Forensic scientists presenting match probabilities can prevent the defence fallacy by framing their evidence as a likelihood ratio and explicitly stating what the LR does and does not represent. The LR quantifies how much more probable the observed evidence is under one hypothesis than under the competing hypothesis. It does not specify the prior probability, and the expert should say so. Combining the LR with the prior is the task of the tribunal, informed by all the evidence in the case.

Judges and counsel can prevent the defence fallacy from misleading juries by requiring experts to state the population assumptions underlying any RMP figure and by directing juries that a match probability must be evaluated alongside the other evidence, not in isolation. Some jurisdictions now have standard jury directions on forensic statistics for this purpose.

• Express the match probability as a likelihood ratio, not as a bare number.
• State explicitly which population was used as the reference database and why.
• Remind the tribunal that the LR must be combined with prior odds derived from other case evidence, and that this combination is not the expert's role.
• In a cold-database-trawl case, advise that the prior probability is lower and the threshold for prosecution should reflect that.
• Avoid presenting match statistics alongside language that implies either near-certain guilt or near-worthless evidence.

Training for legal professionals in forensic statistics is the longer-term solution. The UK's Forensic Science Regulator, the US National Commission on Forensic Science, and equivalent bodies elsewhere have advocated for statistical literacy programmes for judges, prosecutors, and defence counsel. The defence fallacy persists partly because counsel who make the argument and tribunals who accept it lack the statistical grounding to identify the error at the moment it is made. See History of Statistical Evidence in Courts for how courts have evolved in their handling of these issues.