Method Validation and Fitness for Purpose

ISO 17025:2017 is the international standard governing competence in testing and calibration laboratories. Accreditation to this standard is required for forensic laboratories in most jurisdictions that participate in mutual recognition arrangements, including those covered by ILAC (International Laboratory Accreditation Cooperation). The standard's clause 7.2 requires laboratories to validate all methods before use, define the scope of the validation in terms of the measurement procedure and its intended application, and retain records of the validation data.

The validation parameters that ISO 17025 expects to be addressed include: selectivity, range, linearity of response, detection and quantitation limits, precision (repeatability and within-laboratory reproducibility), accuracy or trueness against certified reference materials, measurement uncertainty, and robustness. Not all parameters apply to every method type. A qualitative method that reports a binary present/absent conclusion does not need a linearity assessment, but it does need a detection limit and a false-positive and false-negative rate. A quantitative method needs all of the above.

Accreditation bodies in different jurisdictions implement the standard through their own schemes. In the United Kingdom, UKAS (United Kingdom Accreditation Service) accredits forensic providers under ISO 17025 and publishes forensic-specific supplementary criteria. In the United States, ANAB (ANSI National Accreditation Board) and A2LA perform similar roles. In India, NABL (National Accreditation Board for Testing and Calibration Laboratories) accredits forensic science laboratories and references ISO 17025 as the normative document. The substantive validation requirements are the same across all of these schemes because they all derive from the same international standard.

The limit of detection (LOD) defines the boundary between a true negative and a result that is below detection. It is conventionally estimated from replicate measurements of a blank (a sample known to contain no analyte) by calculating the mean blank signal and adding three standard deviations. A sample producing a signal above this threshold is said to give a detectable response. The choice of three standard deviations corresponds approximately to a 1-in-300 false-positive rate at the detection boundary.

The limit of quantitation (LOQ) is a stricter threshold, typically the blank mean plus ten standard deviations. Above the LOQ, the measurement carries enough signal-to-noise that a numerical value with stated precision can be assigned. Between the LOD and the LOQ, the analyte is present but cannot be reliably quantified. Forensic reports must be explicit about which threshold a result falls above or below: a result between LOD and LOQ should not be reported as a precise concentration.

Forensic significance turns on interpretation. A drug screening test with a high LOD may fail to detect a therapeutic dose, producing a false negative that exonerates when the drug was present. A DNA quantitation tool with an LOD lower than the amount actually present in a sample will flag the sample as suitable for profiling when in reality the profile will be incomplete. Both scenarios represent fitness-for-purpose failures even when the method's own LOD is accurately characterised.

Selectivity testing asks whether the method responds to the target analyte specifically or whether other substances produce the same or a similar signal. In forensic toxicology, a urine immunoassay screen for amphetamines is known to cross-react with certain decongestant medications. The selectivity of the assay is therefore limited, and a positive screen must be confirmed by a method with greater selectivity, typically gas chromatography-mass spectrometry (GC-MS), before a forensic conclusion can be drawn. The validation record should document which interferents were tested, at what concentrations, and what response they produced.

Precision encompasses two distinct concepts. Repeatability is the variation obtained when the same sample is analysed repeatedly by the same analyst on the same instrument under identical conditions in a short period. Reproducibility is the variation observed when conditions change: a different analyst runs the method, a different instrument is used, the sample is analysed on a different day, or a different laboratory runs the same sample. Both matter forensically. Repeatability sets the floor for how consistent a result is under ideal conditions. Reproducibility sets the realistic uncertainty when evidence travels between laboratories or when a result needs to be replicated.

Robustness is evaluated by intentionally varying method parameters within plausible ranges and observing whether performance degrades. A Plackett-Burman experimental design or a one-factor-at-a-time approach is common. Parameters tested might include extraction temperature, pH of the extraction solvent, mobile-phase composition in chromatography, or incubation time in immunoassay. A method that fails its own acceptance criteria when temperature shifts by 2 degrees Celsius has poor robustness, and that fragility must be documented in the validation record and managed operationally through tighter environmental controls.

ISO 17025:2017 and the GUM (Guide to the Expression of Uncertainty in Measurement, published by BIPM) require that every quantitative measurement be accompanied by a statement of its uncertainty. Measurement uncertainty is not the same as error. Error is the difference between a measured value and the true value. Uncertainty is the range within which the true value is expected to lie, at a stated level of confidence, given everything that is known about the measurement process.

A blood alcohol concentration reported as 82 mg/100 mL with a measurement uncertainty of plus or minus 3 mg/100 mL at 95% confidence means the true value is expected to lie between 79 and 85 mg/100 mL in nineteen cases out of twenty. In a jurisdiction where the legal limit is 80 mg/100 mL, the reported value of 82 mg/100 mL does not unambiguously place the true value above the limit. How to handle that ambiguity is a legal and policy question, but the forensic scientist must report the uncertainty so that the question can be addressed honestly.

Uncertainty has two components: Type A uncertainty is estimated from statistical analysis of repeated measurements, corresponding roughly to the standard deviation. Type B uncertainty is estimated from other sources of information, such as the uncertainty stated for a reference material, instrument calibration certificates, or published scientific knowledge about a process. A complete uncertainty budget accounts for both types and combines them using the law of propagation of uncertainty before expressing the combined uncertainty as an expanded uncertainty at a defined coverage probability.

A method is fit for purpose when its validated performance is adequate to answer the specific evidentiary question posed in a particular case. This requires comparing the method's performance profile against the demands of the question. If the question is whether a blood sample contains more than 50 micrograms per litre of a substance and the method's LOQ is 200 micrograms per litre, the method is not fit for that question. If the question is whether two samples share the same source and the method's between-laboratory reproducibility is wide enough that two samples from different sources could easily produce the same result, the method does not discriminate well enough to support a strong likelihood ratio.

The fitness-for-purpose assessment is the bridge between the validation record and the case report. It should appear explicitly in any case where the question at issue approaches the limits of the method's performance. A method that is adequate for routine casework at high concentration may be inadequate for a specific case involving trace quantities or an unusual matrix. The analyst must make this assessment case by case, not assume that accreditation to ISO 17025 implies universal adequacy.

International frameworks support this requirement. The ENFSI Guideline for Evaluative Reporting explicitly links the strength of a likelihood ratio to the quality of the underlying model, which in turn depends on validation data. The ILAC G19 guidelines for forensic science laboratories state that uncertainty and method limitations must be considered when formulating conclusions. Courts in the United Kingdom, United States, and Australia have all encountered cases where evidence was excluded or conviction overturned because the method's fitness for the specific purpose was not adequately demonstrated.

A likelihood ratio for forensic evidence requires the analyst to assign probabilities to the observed result under two competing propositions, typically that the suspect is the source of the trace and that some other unrelated person is the source. Both probabilities depend on a model of how the measurement behaves. That model has parameters: the mean and variance of measurements from same-source comparisons, the mean and variance of measurements from different-source comparisons, and the error rates of the method itself.

Validation data provide those parameters. If the method's inter-laboratory reproducibility standard deviation is large, same-source measurements from the same individual will scatter widely, and so will different-source measurements. The overlap between the two distributions will be large, and the LR will be modest even for genuine matches. A method with tight reproducibility produces distributions with less overlap, and a given degree of similarity between two measurements supports a larger LR. The validation study is, in statistical terms, the characterisation of the distributions that the LR calculation will use.

Error rates require special attention. A method with a known false-positive rate of 1 in 100 cannot support an LR of 10,000 for a match: the claimed discrimination exceeds what the method's error rate permits. This is sometimes called the error rate ceiling on the LR. The validation study must include assessments with samples from known different sources so that the false-positive rate is empirically established, not assumed to be zero. SWGMAT and OSAC standards, the ENFSI evaluative reporting guideline, and the UK Forensic Regulator's Codes of Practice all address this point explicitly. For more on how LR values are constructed and used, see Role of Statistics in Evidence Evaluation and Numbers in Forensic Conclusions.