The Role of Statistics in Evidence Evaluation

Forensic statistics is not a single technique but a set of tools matched to distinct inferential tasks. Understanding which task is in play is the first step to choosing the right statistical approach and avoiding misinterpretation.

Comparison asks whether the measured difference between two samples is smaller than would be expected from measurement error or natural variation alone. Classification asks which group a sample most likely belongs to, given prior knowledge of group characteristics. Source attribution asks whether two samples plausibly came from the same object, person, or location, taking into account how common the observed features are in the population. Evaluative reporting is the communication step: it translates the statistical output into a statement about evidential support that a court can interpret without needing to understand the underlying mathematics.

The distinction between comparison and source attribution matters because two samples can be analytically indistinguishable yet still share a source attribution that is weak, if the features they share are very common in the population. A soil sample matching the control sample in colour and texture is analytically indistinguishable but statistically uninformative if those features are present in 40% of soils from the region. The statistical step is what converts an analytical finding into an evidential weight.

Every forensic statistical statement is ultimately a statement about probability under competing hypotheses. The likelihood ratio is the central quantity. If H1 is the prosecution proposition (for example, the defendant transferred this fibre to the victim) and H2 is the defence proposition (the fibre came from some other source), then the LR is P(evidence | H1) divided by P(evidence | H2). An LR of 1000 means the evidence is 1000 times more probable under H1 than under H2. An LR of 1 means the evidence is equally probable under both and therefore provides no support either way. An LR less than 1 supports H2.

Bayes' theorem connects the LR to the overall probability update. In odds form: posterior odds = prior odds multiplied by the LR. The prior odds represent the probability of H1 relative to H2 before the forensic evidence is considered. Multiplying by the LR gives the posterior odds after the evidence. This is the logically correct way to incorporate forensic evidence, and it makes clear that the scientist's job is to provide the LR, while the fact-finder supplies the prior odds based on all other evidence in the case.

In practice the LR is often computed from a population database. For a DNA profile, the numerator P(evidence | H1) is typically 1 if the defendant is the source, because you would certainly observe the defendant's profile if the defendant left the sample. The denominator P(evidence | H2) is the random match probability: how often would a randomly chosen unrelated person produce the same profile? Large population databases, such as those maintained by the FBI's CODIS system or the UK National DNA Database, allow very precise RMP estimates for standard STR profiles. For less well-studied trace evidence types the databases are smaller and the estimates less precise.

See Conditional Probability and Independence for the mathematical foundations, and Random Match Probability for how RMP is derived from population databases.

Two fallacies appear repeatedly in trial transcripts, expert reports, and press coverage of forensic cases. Both arise from confusing conditional probabilities that run in opposite directions.

The prosecutor's fallacy treats P(evidence | innocent) as if it were P(innocent | evidence). An analyst who says the DNA match probability is 1 in 10 million and then implies this means the defendant has only a 1-in-10-million chance of being innocent has made this error. The probability of an innocent person matching is very small, but the probability of guilt given a match depends on how many other potential sources exist and on all the other evidence in the case. The UK Court of Appeal identified this fallacy in R v Doheny and Adams (1997) and it remains a recognised ground of appeal in many common-law systems.

The defence fallacy runs the other direction: treating the existence of innocent explanations as equivalent to the evidence being uninformative. A defence argument that says there are millions of people who could match, so the evidence means nothing overstates the ambiguity. Even with one match in 10 million, a population of 100 million would yield about 10 expected matching individuals; the evidence still substantially narrows the field. The LR framework handles both fallacies correctly because it always expresses the evidence as a ratio rather than as a probability of the ultimate issue.

Evaluative reporting is the practice of expressing a forensic conclusion in terms of the relative support the evidence provides for two explicitly stated competing propositions. It replaces categorical language such as it is the opinion of this examiner that the samples have a common source with a statement of the form: the evidence is strongly more probable if the samples share a common source than if they originate from different sources drawn from the relevant population.

The European Network of Forensic Science Institutes (ENFSI) published a guideline on evaluative reporting in 2015 that sets out the framework most European laboratories use. The guideline requires: explicit statement of the propositions considered, the level of the hierarchy of propositions addressed (source, activity, offence), the likelihood ratio assigned, the population and database underpinning the LR, and any assumptions made. The UK Forensic Science Regulator's Codes of Practice and Conduct mandate the same approach for accredited laboratories in England and Wales. Australian and New Zealand guidelines follow similar principles.

The proposition hierarchy matters. A source-level proposition asks whether two samples share a common physical origin. An activity-level proposition asks whether the defendant performed the action alleged, given that the trace was found where it was. An offence-level proposition asks whether the defendant committed the offence. Scientists are generally equipped to address source-level propositions and sometimes activity-level propositions, but offence-level propositions require legal and factual knowledge beyond the scientist's expertise. Mixing levels is a common error in evaluative reporting that courts have criticised.

Verbal scales convert numeric LR values into descriptive phrases for court communication. ENFSI's recommended scale runs from limited support through moderate, moderately strong, strong, very strong, and extremely strong support, with each tier corresponding to a LR range. The scale is a communication tool, not a substitute for the numeric LR, and the underlying number should always be available for scrutiny.

Every forensic method produces errors. A classification method that assigns a questioned document to one of two authors will sometimes assign it to the wrong author. A DNA profile interpretation protocol will occasionally call a present allele absent (drop-out) or an absent allele present (drop-in). The frequency of these errors in a population of realistic casework samples, measured under casework conditions, is the empirical error rate of the method.

Validation studies establish whether a method works as claimed. A valid method must be shown to produce low error rates on samples whose true answer is known, to perform consistently across operators and laboratories, and to degrade in predictable ways as sample quality decreases. The 2016 PCAST report in the United States reviewed the validation evidence for several forensic feature-comparison disciplines including firearms, bite marks, and footwear, and found that many lacked adequate foundational validity data. The report did not conclude the methods were worthless, but it did conclude that claims of high discriminating power were not supported by the published validation literature.

Statistics cannot substitute for a weak evidential foundation. A sophisticated Bayesian network built on unvalidated feature measurements will produce precise-looking numbers with wide underlying uncertainty that the precision conceals. The discipline requirement is that every input to the statistical model, the feature measurements, the population database, the error rate, must itself be empirically grounded. When inputs are missing or uncertain, the output uncertainty must be propagated and communicated, not hidden behind a point estimate.

The quality of a forensic probability estimate is bounded by the quality of the population database underlying it. Three parameters matter: size, composition, and sampling method. A DNA reference database of 10 000 unrelated individuals from a single ethnic group will produce RMP estimates that are unreliable for individuals from different ethnic backgrounds, because allele frequencies differ between populations. The standard approach in jurisdictions with mixed populations, including the United States, Australia, and India, is to compute the RMP for each major population group and report the least favourable (highest) figure, or to compute a weighted average.

Sampling method affects whether the database is representative. A database assembled from convicted offenders is not representative of the general population in the same way a random household survey would be, because the offender population has demographic characteristics that differ from the general population. For forensic disciplines where the relevant population is all people who could plausibly be the source, a database that over-represents certain groups will produce biased probability estimates.

For some trace evidence types, no published population database exists, or the available databases are very small. Glass refractive index databases, fibre colour databases, and soil mineral composition databases are all smaller and less standardised than DNA reference databases. In these cases the scientist must be explicit about the database used, its size, how it was assembled, and how the absence of a larger database affects confidence in the probability estimate. Overstating precision when the database does not support it is a form of scientific misrepresentation.

See Population Databases for Forensic Statistics for a detailed treatment of database construction and its effect on probability calculations.