Evaluative Opinion Standards and Frameworks

The European Network of Forensic Science Institutes published its Guideline for Evaluative Reporting in Forensic Science in 2015, following a decade of discussion among member laboratories about how to standardise the expression of uncertainty and inferential weight. The guideline applies to all forensic disciplines that produce evaluative opinions: DNA, fingerprint, questioned documents, fire investigation, toxicology, and many others. Its core requirement is that evaluative opinions be expressed as likelihood ratios or, where a numerical LR is not available, as verbal equivalents drawn from a defined scale.

The ENFSI verbal scale has six levels of support for the prosecution proposition: weak (LR between 1 and 10), moderate (10 to 100), moderately strong (100 to 1000), strong (1000 to 10,000), very strong (10,000 to 1,000,000), and extremely strong (above 1,000,000). The scale is one-sided: support for the defence proposition is expressed symmetrically in the opposite direction. Every band has a corresponding verbal phrase that is used in the written report and, where the expert gives oral evidence, in testimony.

The guideline also specifies the structure of a compliant evaluative report. The report must: state the items examined and the methods used; set out the prosecution and defence propositions that frame the evaluation; describe the population model or reference database used to estimate the LR; report the LR value or verbal equivalent; state the key assumptions; and list limitations that could affect the conclusion. This structure makes the opinion auditable. A reviewer can check each step independently.

ILAC G19, most recently updated in 2014, is titled 'Modules in a Forensic Science Process' and addresses the full chain from evidence collection through laboratory analysis to reporting. Its provisions on measurement uncertainty are the most directly relevant to evaluative opinions. The document requires that forensic laboratories identify and quantify the sources of uncertainty in any measurement or analytical result that feeds into an evaluative conclusion.

Measurement uncertainty in this context is not about philosophical doubt: it is the quantified range within which the true value of a measurement is expected to lie, given the method and conditions used. For a breath alcohol analyser, this might be expressed as a concentration value plus or minus a stated margin. For a questioned document examination relying on ink chromatography, it is the reproducibility of the chromatographic profile and the discriminating power of the comparison method. ILAC G19 requires that this uncertainty be propagated through to the evaluative conclusion.

ILAC G19 also addresses the scope of the laboratory's role. It distinguishes between the 'case assessment' stage, where the scientist helps investigators understand what questions the evidence can address, and the 'evaluation' stage, where the analysis and opinion are produced. This distinction matters for accreditation: a laboratory must have documented procedures for both stages, and the opinion produced at the evaluation stage must be traceable to the case assessment decisions made at the earlier stage.

ISO 17025 (most recently revised in 2017) is the foundational standard for any laboratory that generates data used in decision-making. For forensic science, it creates specific requirements in the 'reporting of results' clause. Clause 7.8 of the 2017 standard addresses what a test report must contain. When a laboratory issues an opinion or interpretation, the report must clearly identify it as an opinion, state the basis on which it was formed, and be signed by the qualified person responsible for it.

Accreditation to ISO 17025 is granted by national accreditation bodies. The UK Accreditation Service (UKAS) accredits UK forensic laboratories, the ANSI National Accreditation Board (ANAB) covers many US laboratories, and DAkkS covers German laboratories. The accreditation process includes a technical assessment of the methods used, a review of the laboratory's uncertainty estimation procedures, and inspection of sample reports. A laboratory that holds accreditation has had its evaluative opinion procedures verified by an external body against these requirements.

Drawing together the requirements of the three documents, a compliant evaluative opinion has a defined structure. The following elements are mandatory:

• Items examined: a precise description of every item received, its condition on receipt, and the sub-samples or traces analysed.
• Methods: the validated analytical methods applied, with a reference to the validation data or accredited procedure.
• Proposition pair: the prosecution and defence propositions (Hp and Hd) that frame the evaluation, stated in plain language before the LR is given.
• Likelihood ratio or verbal equivalent: the numerical LR where the method supports it, or the appropriate verbal level from the ENFSI scale if a numerical value is not available.
• Assumptions: the key assumptions on which the LR depends, including the reference population chosen and any assumptions about the transfer and persistence of trace material.
• Limitations: conditions that could reduce the reliability of the conclusion, such as degraded samples, small reference databases, or conditions that fall outside the validated range of the method.

A non-compliant opinion typically fails in one of three ways. It makes a source attribution ('this blood is the victim's'). It conflates the likelihood of the evidence with the probability of guilt. Or it reports an LR without disclosing the assumptions on which it rests, making it impossible for the court, the opposing expert, or a reviewing scientist to evaluate whether the figure is reliable.

The prosecutor's fallacy is the most common form of non-compliance: inverting the conditional probability so that P(evidence | hypothesis) is presented as P(hypothesis | evidence). A compliant evaluative opinion, by requiring that the LR be separated from the posterior probability and that the jury be told explicitly that the prior is theirs to assess, structurally prevents this error.

Accreditation under ISO 17025 and compliance with ENFSI or ILAC G19 do not determine whether evidence is admissible. Admissibility is a legal question decided under the procedural rules of the relevant jurisdiction. In the United States, the Daubert standard (established in Daubert v. Merrell Dow Pharmaceuticals Inc., 1993) requires trial judges to assess whether scientific testimony is based on sufficient facts, a reliable methodology, and has been applied reliably to the facts of the case. Frye, still applied in some US states, asks whether the method is generally accepted in the relevant scientific community.

In England and Wales, the Criminal Procedure Rules Part 19 and the Criminal Practice Directions set out what expert reports must contain and require that experts understand their overriding duty to the court. The Law Commission's 2011 report on expert evidence recommended a reliability test modelled loosely on Daubert, and while Parliament did not enact the full proposal, courts have increasingly scrutinised whether forensic methods are adequately validated. In India, the Bharatiya Sakshya Adhiniyam 2023 governs the admissibility of expert opinion evidence, continuing the general principle from prior law that expert opinion is admissible where the court needs assistance on a matter of specialised knowledge; the statute does not specify validation requirements, but the quality of the underlying method is a factor courts may examine. The European Court of Human Rights has addressed the fairness of expert evidence under Article 6 of the European Convention on Human Rights in several cases, focusing on whether the defendant had an adequate opportunity to challenge the evidence.

The practical effect is that accreditation and standards compliance are powerful but not decisive. A laboratory with ISO 17025 accreditation and ENFSI-compliant reports is far better placed to withstand a Daubert challenge or a Criminal Procedure Rules objection than one without these credentials. But accreditation does not bind the court, and an admissible report from an accredited laboratory can still be rejected by a jury on the merits of the evidence.

The transition to standards-compliant evaluative reporting has not been uniform across jurisdictions or disciplines. In fingerprint science, some practitioners and professional bodies in the United States have resisted the shift from categorical identification language ('this fingerprint belongs to the suspect') to likelihood ratio language. The argument is partly cultural, rooted in longstanding courtroom practice, and partly technical: validated population databases for fingerprint comparison are less developed than those for DNA, making numerical LRs harder to defend.

In DNA, the transition happened earlier and is more complete. Short tandem repeat (STR) profile matching has had validated population databases since the 1990s, and the random match probability has been a standard feature of DNA evidence for nearly as long. The debate in DNA has moved to more challenging evidence types: low template DNA, mixed profiles, and familial searching, where the uncertainty is greater and the LR calculation more contentious.

Pattern-based disciplines such as toolmark and bite-mark comparison face the most difficulty. The PCAST report found insufficient foundational validity for bite-mark analysis and limited validity for toolmark analysis, meaning that the validation requirement of ISO 17025 is a genuine barrier to compliance rather than a formality. Disciplines that cannot demonstrate foundational validity through published, reproducible studies cannot produce compliant evaluative opinions: they can report observations, but they cannot quantify the weight of those observations without the required validation data.