Likelihood Ratios and Verbal Scales in Geological Evidence

The likelihood ratio is simply the probability of the observed evidence given the prosecution hypothesis divided by the probability of the observed evidence given the defence hypothesis. Written out, that is P(E | Hp) / P(E | Hd). Each term requires a different probability estimate, and those estimates come from different parts of the data.

For a soil geochemical comparison, P(E | Hp), the probability of seeing this degree of similarity given the two samples come from the same source, is estimated from within-source variation. The analyst collects replicate samples from the reference location (or uses the spread around a centroid in PCA space) to characterise how much variation is normal within a single source. If the questioned sample falls well within that spread, the numerator is high.

P(E | Hd), the probability of seeing this degree of similarity given the samples come from different sources, is estimated from between-source variation in the reference population. The analyst asks: among all the different-source pairs I can form from my reference collection, how often do two soils from different locations look as similar as the two in this case? If rare, the denominator is low, and the LR is large.

Calculating an LR and knowing whether it is reliable are distinct problems. Calibration is the process of checking that an LR calculation method produces values that actually correspond to the stated frequency difference between the two hypotheses. A method that reports LR = 10,000 for cases that are actually only 100 times more common under the same-source hypothesis is miscalibrated and misleading to a court.

The Tippett plot provides a visual calibration check. The analyst takes a large set of known same-source pairs and a large set of known different-source pairs, calculates the LR for each, and plots their cumulative distributions on the same axes. A well-calibrated system produces two curves that diverge sharply: the same-source distribution concentrates at high LR values, the different-source distribution concentrates at LR values below 1. If the curves overlap substantially, the method is producing ambiguous LRs in the region of overlap, and that ambiguity should be reported as a limitation.

The ENFSI verbal equivalence scale was developed to provide a common vocabulary for expert witnesses reporting LR-based evidence conclusions across forensic disciplines. The scale maps LR ranges to verbal descriptors, with LR values above 1 supporting the same-source hypothesis and values below 1 (or equivalently, LR values as 1/x supporting the defence hypothesis).

The scale is not mandatory, and individual jurisdictions have adopted variations. The UK Forensic Science Regulator's guidance broadly follows the ENFSI scale. The scale is symmetric: an LR of 0.001 (i.e., 1/1,000) would be reported as strong support for the defence hypothesis.

The documented problem with verbal scales is inconsistency. Studies by Champod and Vuille, and by others in the DNA context, found that practitioner choice of verbal term for a given LR varies considerably, and that jurors interpret the same phrase differently depending on the strength of other evidence in the case. For geological evidence specifically, there is very limited empirical data on how jurors process verbal LR descriptions, and this is an open research gap.

A full soil comparison typically generates multiple test results: colour (Munsell notation), particle-size distribution, mineralogical composition by polarising microscopy, and elemental geochemistry by ICP-MS. Each test gives an LR. The question is whether they can be combined.

The rule for combining LRs by multiplication holds only when the tests are statistically independent. Independence means that knowing the outcome of one test gives no information about the outcome of another, given either hypothesis. Colour reflects both the iron-oxide content and the organic-matter content of a soil; particle size reflects its depositional and weathering history. These are largely controlled by different processes and are often approximately independent. Two elemental concentrations measured in the same ICP-MS run are almost certainly correlated, driven by co-occurring minerals, and should not be treated as independent LRs.

1. Assess independence
   Run a PCA or correlation matrix on the reference population data. Tests measuring correlated properties should be combined into a single multivariate LR rather than multiplied as separate values.
2. Calculate LR per independent test
   Estimate each LR separately using the within-source and between-source variation for that test. Report the individual values before combining.
3. Multiply independent LRs
   Where independence has been justified, multiply. A colour LR of 20, a particle-size LR of 15, and a mineralogy LR of 40 combine to approximately 12,000, which falls in the 'strong support' range of the ENFSI scale.
4. Report with uncertainty
   The combined LR inherits the uncertainty of each component estimate. Report the approximate value and note the key assumptions, particularly the independence assumption and the size of the reference population.

The practical value of the ENFSI verbal scale depends on two kinds of consistency: consistency among scientists reporting evidence, and consistency among jurors and judges interpreting it. Both have been challenged in the empirical literature.

On the scientist side, studies in DNA and fingerprint evidence have shown that practitioners given identical numerical LRs disagree on which verbal phrase to apply, particularly near scale boundaries. A practitioner who calculates LR = 950 may call it 'moderately strong support'; another may round up to 'strong support'. The effect on the court's assessment of the evidence depends on how sensitive jurors are to the verbal distinction.

On the juror side, the phrase 'strong support' has been shown in mock-jury studies to be interpreted as a much higher probability statement than the LR value warrants, especially when the surrounding case facts are already incriminating. Jurors can read 'strong support for the prosecution hypothesis' as close to certainty, when an LR of 5,000 with a prior that depends on all other case evidence might still leave meaningful doubt. This is not a problem with the LR itself, it is a communication problem, and it has not been solved.

The following numerical example uses a univariate lead concentration for clarity; the same logic extends to multivariate data via kernel density estimation or parametric distributions.

The reference sample (crime scene) has a lead concentration of 185 mg/kg. The questioned sample (suspect's boot) has a lead concentration of 192 mg/kg. Multiple samples from the crime-scene location (within-source replicates) have a mean of 183 mg/kg and a standard deviation of 12 mg/kg. The reference population (25 samples from surrounding areas) has a mean of 94 mg/kg and a standard deviation of 61 mg/kg.

1. Numerator: the probability of observing a value of 192 mg/kg given the within-source distribution N(183, 12). Using a normal probability density function, this evaluates to approximately 0.027.
2. Denominator: the probability of observing a value of 192 mg/kg given the reference population distribution N(94, 61). Using the same function, this evaluates to approximately 0.0022.
3. LR: 0.027 / 0.0022 = approximately 12. This falls in the 'moderate support' category on the ENFSI scale.
4. Interpretation: lead concentration alone provides only moderate support for the same-source hypothesis. Lead is elevated in both samples relative to the area background, which helps, but the within-source and between-source distributions overlap considerably. If mineralogy and particle size provided independent LRs of 30 and 25 respectively, the combined LR would be 12 x 30 x 25 = 9,000, placing the combined evidence firmly in 'strong support'.