Measurement Uncertainty and Proficiency Testing

The GUM distinguishes between the error in a measurement (the difference between the measured value and the true value, which is never exactly known) and the uncertainty (a quantified interval that characterises the spread of values that could reasonably be attributed to the measurand). This distinction matters in court: a laboratory does not claim to know the true value, it claims that the true value lies within the uncertainty interval with a stated probability.

Type A evaluation applies when repeated measurements of the same quantity under the same conditions are available. The standard uncertainty is the standard deviation of the mean: u_A = s / sqrt(n), where s is the standard deviation of the individual observations and n is the number of replicates. Blood alcohol analysis, drug weighing, and DNA quantification all generate replicate data suitable for Type A evaluation. The reliability of u_A improves as n increases, and laboratories should document the minimum n required before a Type A estimate is considered stable.

Type B evaluation covers all sources that cannot be assessed by direct statistical analysis of repeated measurements. A calibration certificate for a reference standard gives an expanded uncertainty U_cert and a coverage factor k; the standard uncertainty is U_cert / k. A balance specification quoting a maximum permissible error of plus or minus 0.1 mg is treated as a rectangular distribution with half-width a = 0.1 mg and standard uncertainty a / sqrt(3). The assignment of a probability distribution to a Type B source is a professional judgement, and laboratories must document the reasoning.

Most forensic measurements are not direct: the reported result is a function of several inputs, each with its own uncertainty. The GUM law of propagation of uncertainty states that for a result y = f(x_1, x_2, ..., x_n) where the inputs are independent, the combined standard uncertainty is:

u_c(y) = sqrt[ sum_i (df/dx_i)^2 * u(x_i)^2 ]

The partial derivative df/dx_i is called the sensitivity coefficient for input x_i. It converts the uncertainty in that input into its contribution to the output uncertainty. For a product y = x_1 * x_2, the relative combined uncertainty is sqrt[ (u_r(x_1))^2 + (u_r(x_2))^2 ], where u_r denotes relative (fractional) standard uncertainty. This additive-in-quadrature rule means that a large uncertainty in one input dominates if the others are small, and doubling precision on minor contributors has little effect.

When inputs are correlated, cross-product terms must be added. Correlation arises when the same reference standard is used to calibrate a series of measurements, or when the same balance is used to weigh both sample and tare. In practice, forensic procedures are often designed to minimise correlation, and many laboratories assume independence as an approximation, but the approximation should be documented and its validity confirmed.

See Confidence Intervals and What a Sample Supports for a complementary treatment of how interval estimates are interpreted in evidential contexts.

The combined standard uncertainty u_c is a one-sigma equivalent. For reporting and court purposes, laboratories report the expanded uncertainty U = k * u_c, where k is chosen to achieve a desired coverage probability. For a normal (Gaussian) output distribution, k = 2 covers approximately 95.45 percent and k = 2.576 covers 99 percent. Most accredited forensic laboratories report at k = 2 unless a higher level is required by procedure or statute.

The coverage probability is only meaningful if the output distribution is well characterised. When the combined uncertainty has few degrees of freedom (because Type A components come from small replicate sets), the effective degrees of freedom should be estimated using the Welch-Satterthwaite formula, and the coverage factor taken from the t-distribution at the corresponding degrees of freedom rather than from the normal distribution. Laboratories with n = 3 replicates face substantially wider coverage factors than those with n = 20 or more.

Different jurisdictions handle the disclosure obligation differently. Under the UK Crown Prosecution Service Forensic Science Regulator's Codes of Practice, accredited laboratories must report expanded uncertainty at 95 percent and state the coverage factor. US federal guidelines for forensic DNA laboratories (FBI Quality Assurance Standards) require uncertainty documentation at the method-validation stage, with results expressed consistently with validated procedures. Under the Bharatiya Sakshya Adhiniyam 2023, expert opinion evidence carries the obligation of the expert to state the basis of the opinion, which includes the precision of any measurement on which the opinion rests.

Proficiency testing schemes are organised by external bodies: UKAS-approved providers in the UK, A2LA-affiliated providers in the US, NABL-affiliated providers in India, and bodies operating under ILAC mutual recognition agreements across the EU. An organiser prepares or sources materials of known or consensus composition, distributes them blind to participating laboratories, collects results, calculates the assigned value and the standard deviation for proficiency assessment (s_PT), and scores each result.

The z-score is the standard scoring statistic defined in ISO 13528: z = (x - X) / s_PT. The assigned value X may be the certified value of a reference material, the consensus median of participants, or a value obtained from a reference laboratory. The s_PT is typically set using a fitness-for-purpose approach: it reflects the standard deviation that would be tolerable given the end use of the measurement, not simply the average performance of participants. Setting s_PT too wide allows systematic errors to pass undetected; setting it too narrow fails well-performing laboratories.

ISO 13528 defines performance categories: |z| ≤ 2 is satisfactory, 2 < |z| < 3 triggers an action signal, and |z| ≥ 3 is unsatisfactory. Two consecutive unsatisfactory results in the same analyte or method typically trigger a mandatory investigation under most accreditation body rules. Some schemes also compute the zeta-score (z'), which incorporates the laboratory's own stated uncertainty alongside s_PT, allowing the score to reflect the realism of the laboratory's uncertainty claim.

An evaluative opinion translates a measurement result into an inference about a case proposition. The uncertainty of the measurement is not separate from that inference: it directly affects the likelihood ratio or probability assigned to competing propositions. For a finding such as 'the blood alcohol concentration was 0.095 g/100 mL', where the legal limit is 0.08 g/100 mL, an expanded uncertainty of 0.010 g/100 mL means the interval [0.085, 0.105] g/100 mL is consistent with the data at 95 percent probability.

Some jurisdictions have adopted a worst-case convention for threshold enforcement: the reported value is the measured value minus the expanded uncertainty, so the laboratory asserts only what the data can support at the lower confidence bound. The UK Road Traffic Act enforcement practice applies this convention for evidential breath testing. Other jurisdictions report the central value and require the court to apply the uncertainty interval when drawing inferences. Neither approach changes the underlying science; the difference is where the risk of error is allocated between prosecution and defence.

The Role of Statistics in Evidence Evaluation topic covers the broader framework of how numerical evidence feeds into the court's decision. See Role of Statistics in Evidence Evaluation for that context. At the reporting stage, the practical requirements are: state the result, state the expanded uncertainty, state the coverage factor and probability, and explain in plain language what the interval means for the specific question the court is deciding.

Accreditation under ISO/IEC 17025 requires documented uncertainty procedures, PT participation, and uncertainty disclosure in test reports. The ILAC Policy on Measurement Uncertainty in Calibration (ILAC P14) and the EURACHEM/CITAC guide 'Quantifying Uncertainty in Analytical Measurement' (CG4) provide additional implementation guidance used in forensic laboratories across Europe, the UK, India, and countries that have adopted ISO-aligned accreditation frameworks.

National accreditation bodies mandate PT participation as a condition of ISO/IEC 17025 accreditation. In the UK, UKAS requires forensic laboratories to participate in appropriate PT schemes at a frequency commensurate with the test volume and risk level. In the US, the FBI's Quality Assurance Standards for forensic DNA testing require PT twice annually, with results reviewed by technical reviewers and reported to the laboratory director. In India, NABL (National Accreditation Board for Testing and Calibration Laboratories) requires PT participation under its accreditation criteria aligned to ISO/IEC 17043.

PT results can be disclosed in legal proceedings. Defence teams in several jurisdictions have successfully obtained PT records through discovery procedures and used patterns of unsatisfactory performance to challenge the reliability of a laboratory's results in a specific case. Courts in the US (under Daubert and related standards), the UK (under Criminal Procedure Rules), and the EU (under national procedural laws implementing Directive 2016/343) have addressed the admissibility and weight of PT evidence as part of reliability challenges to forensic evidence.

A single unsatisfactory PT result does not automatically invalidate case results produced during the same period, but it triggers an obligation to investigate whether the PT failure represents a systemic problem affecting case work, and to disclose that investigation to the court if it does. Laboratories should have documented procedures specifying the steps taken when PT performance is unsatisfactory, including retrospective review of affected case results, root cause analysis, and corrective action before the next testing round.