Quality Systems: ISO 17025, NABL, ASCLD-LAB and Proficiency Testing

ISO/IEC 17025, currently in its 2017 edition (replacing the 2005 edition that most laboratories spent the previous decade building toward), is the international standard for the competence of testing and calibration laboratories. It applies to any laboratory that wishes to demonstrate that it operates competently and is capable of generating technically valid results. Its requirements are organised into two main sections: management requirements (governing the operational structure of the laboratory) and technical requirements (governing the quality of the test results themselves).

For forensic document examination laboratories, the management requirements of ISO 17025 address impartiality (the laboratory must identify threats to its impartiality and manage them); confidentiality (information about casework must be protected); document control (every SOP, reference table, and procedural document must be version-controlled so that the laboratory can demonstrate which version was in use on a given date); and complaints handling. The impartiality requirement has direct relevance to the cognitive bias discussion: a laboratory that has financial or institutional pressure to reach conclusions favourable to one party faces an accreditation risk that ISO 17025 specifically requires it to manage.

The technical requirements address method selection and validation, sampling procedures, the handling and preparation of test items, measurement uncertainty, and reporting of results. For document examination, the critical technical requirement is method validation. ISO 17025 requires that methods be validated to demonstrate that they are fit for purpose. For handwriting comparison, this means demonstrating (through internal validation studies, published literature, or proficiency-test data) that the comparison methodology, as applied by the laboratory's examiners, reliably discriminates between authors under the conditions in which it is applied. The 2017 edition strengthened this requirement compared to the 2005 edition, particularly for methods that produce qualitative rather than quantitative results, which includes most handwriting and document examination methods.

Uncertainty of measurement, a quantitative concept from measurement science, applies awkwardly to qualitative forensic examinations. The 2017 edition of ISO 17025 addressed this by requiring laboratories to identify the factors that contribute to uncertainty in qualitative methods and to document how these are managed. For handwriting examination, contributing factors include the quality of the questioned writing, the quantity and comparability of the known standards, and the examiner's own experience and bias-management procedures.

The National Accreditation Board for Testing and Calibration Laboratories (NABL) operates under the Department for Promotion of Industry and Internal Trade (DPIIT), Government of India, and is India's national accreditation body for testing and calibration laboratories. NABL grants accreditation against ISO/IEC 17025, supplemented by specific technical criteria documents for particular laboratory types.

For forensic science laboratories, the relevant criteria are contained in NABL Technical Document T-126: "Specific Criteria for Accreditation of Forensic Science Laboratories." T-126 defines additional requirements beyond the general ISO 17025 framework that are specific to forensic work, including chain-of-custody handling of evidential items, documentation of examination findings in a form that can be disclosed in court proceedings, and personnel qualification requirements for forensic examiners. T-126 covers multiple forensic disciplines; the document examination sections address the instruments and methods to be validated, the reference materials and databases to be maintained, and the proficiency testing obligations for document examination units.

The Central Forensic Science Laboratories (CFSLs) at New Delhi, Kolkata, Hyderabad, and Chandigarh operate under the Union Home Ministry through the Directorate of Forensic Science Services (DFSS) and maintain NABL accreditation for various disciplines. The document examination divisions at these laboratories have historically been among the more robustly accredited units because document examination work goes to higher courts and faces sustained expert challenge more frequently than some other forensic disciplines. State forensic science laboratories (FSLs) present a more varied picture: several large state FSLs (Maharashtra, Tamil Nadu, Andhra Pradesh) maintain NABL accreditation for their document examination units, while smaller state FSLs may operate without current accreditation, providing examiner opinions that are technically admissible under BSA 2023 s.39 but that lack the independent quality assurance that accreditation provides.

The American Society of Crime Laboratory Directors Laboratory Accreditation Board (ASCLD-LAB) was the primary forensic laboratory accreditation body in the United States from 1981 until its merger with the American National Standards Institute National Accreditation Board (ANAB) in 2016. ASCLD-LAB developed its own accreditation programme, known as the ASCLD-LAB Legacy program, which was tailored specifically to forensic science laboratories. The Legacy programme assessed laboratory management, personnel, equipment, evidence handling, methods, and reporting, using criteria developed by forensic-science practitioners rather than being derived directly from ISO 17025.

The National Academy of Sciences 2009 report, which analysed weaknesses in the US forensic science system, specifically recommended that forensic laboratories seek accreditation through ISO 17025-based programmes. This contributed to ASCLD-LAB's development of an ISO 17025-aligned International programme (run alongside the Legacy programme), and ultimately to the merger with ANAB and the discontinuation of the Legacy programme. As of 2016, US forensic laboratories seeking accreditation through ANAB are assessed against ISO/IEC 17025 with supplementary forensic-specific requirements, including the OSAC-approved standards that NIST's Organisation of Scientific Area Committees develops for individual forensic disciplines.

For document examination laboratories in the US, the ANAB/ISO-17025 transition imposed more rigorous method-validation and measurement-uncertainty documentation requirements than the ASCLD-LAB Legacy programme had required. Laboratories that had operated with informal or undocumented comparison procedures needed to formalise those procedures as validated SOPs, with defined acceptance criteria, training requirements for personnel performing the method, and documented verification of competence. Some smaller state and local document examination units found this transition difficult and have operated without ANAB accreditation, relying on Daubert / Frye admissibility without accreditation-based quality assurance.

The FBI Questioned Documents Unit and the US Secret Service Forensic Services Division have maintained accreditation and have developed internal quality management systems that exceed the minimum requirements of ISO 17025, including comprehensive proficiency testing programmes and internal validation studies for specific comparison methods.

The Forensic Science Regulator was established in England and Wales in 2008 as a non-statutory body responsible for setting quality standards for forensic science providers operating in the criminal justice system. For its first thirteen years, the Regulator could issue Codes of Practice and guidance but had no statutory power to enforce compliance. The Forensic Science Regulator Act 2021 changed this, giving the Regulator statutory power to require compliance, to investigate providers who do not meet the required standards, and to publish compliance decisions.

The Regulator's Codes of Practice and Conduct (the "Codes") cover the full range of forensic science disciplines and set out the quality standards providers must meet. For document examination, the relevant provisions address: the competence and training of examiners; the validation of comparison methods; the use of calibrated instrumentation; the management of cognitive bias through information-management protocols; the format of expert reports; and the conduct of verification. The Codes also include specific provisions on digital forensics, imaging, and the examination of electronic documents, reflecting the expanding scope of document examination work.

UKAS (United Kingdom Accreditation Service) is the national accreditation body for the UK, operating under a Memorandum of Understanding with the government and recognised by the European co-operation for Accreditation (EA). UKAS accredits forensic science providers against ISO/IEC 17025. The FSR Codes require that providers working on cases in the criminal justice system in England and Wales be accredited by UKAS for the specific activities they perform.

The 2023 update to the FSR Codes introduced more specific requirements for evaluative reporting, requiring that expert opinions be expressed in a form that reflects the strength of the evidence rather than simply the direction, and that evaluative opinions clearly state the hypotheses under evaluation, consistent with the ENFSI evaluative reporting framework. Document examiners working in England and Wales are therefore expected to be moving toward LR-framed or at least hypothesis-explicit conclusion language, even in the absence of fully computed numerical LR values for handwriting examination.

Proficiency testing is the mechanism by which a laboratory's performance is assessed using materials and tasks that are comparable to real casework but where the outcome is known to the test provider, not to the laboratory being tested. There are two types: open proficiency tests, in which the examiner knows they are being tested; and blind proficiency tests, in which the examiner handles the test material as if it were casework and does not know it is a proficiency exercise. The PCAST 2016 report specifically called for more blind proficiency testing in forensic science because open testing, where examiners know they are being evaluated, may produce performance data that overestimates real-casework performance.

In the United States, Collaborative Testing Services (CTS) has provided proficiency testing to forensic science laboratories since 1978. CTS operates in multiple forensic disciplines, including document examination. The CTS document examination proficiency tests present participating examiners with questioned writing samples and known standards, asking them to reach conclusions using their normal methodology. The results are compiled and reported anonymously, so each laboratory can compare its performance against the aggregate. CTS tests are open-proficiency tests; examiners know they are being tested. The FBI and many ANAB-accredited state laboratories participate in CTS proficiency testing as part of their accreditation obligations.

The ENFSI (European Network of Forensic Science Institutes) proficiency testing programme covers the Forensic Document Examiners Working Group. ENFSI proficiency tests are conducted on a rotating schedule across member laboratories in EU member states and several associated countries. The ENFSI tests include handwriting comparison, signature examination, and ink differentiation exercises. Results are compiled by the ENFSI working group and used to identify outlier laboratories whose performance deviates from the group, prompting quality improvement follow-up.

The OSAC (Organisation of Scientific Area Committees) validation studies programme, operating under NIST, is focused on developing the black-box examiner-performance studies that PCAST 2016 recommended as missing from the handwriting examination literature. The NIST 2020 "Forensic Handwriting Examination and Human Factors" report, co-authored with OSAC input, identified specific study designs that would address the PCAST gap. These studies, when conducted, will provide the population-level examiner error-rate data that the SWGDOC conclusion scale currently lacks.

Document control, in the ISO 17025 sense, means the systematic management of all documents that describe or affect the laboratory's procedures, so that the current approved version is the one in use, all previous versions are archived with their dates of validity, and changes to procedures are reviewed and authorised before implementation. For forensic document examination laboratories, the documents under control typically include: SOPs for each comparison method (handwriting comparison, signature examination, VSC operation, ESDA operation, photocopy examination); reference tables for conclusion scale positions and their definitions; equipment calibration schedules and records; and training and competence records for each examiner.

The importance of document control becomes apparent when a conclusion is challenged years after the examination was conducted. The defence advocate asks: what SOP was in use when the examination was conducted? Was that version approved? Had it been superseded by a revised version that would have changed the procedure? Was the instrument used calibrated, and was the calibration current? These questions can be answered definitively only by a laboratory with an effective document-control system. A laboratory without document control may be unable to establish, retrospectively, what procedure was actually followed.

Method validation is the structured process by which a laboratory demonstrates that a method is fit for the purpose for which it is being used, under the conditions in which it will be applied. For document examination, validation data might include: results from proficiency tests showing that the method produces correct conclusions when applied to materials with known answers; internal comparison studies conducted by laboratory examiners on materials of known authorship; and published literature demonstrating the discriminating capability of specific feature analysis approaches.

The question of how non-accredited opinion is treated in court varies by jurisdiction. In England and Wales, following the FSR Act 2021, a forensic science provider who is not UKAS-accredited for the activities they perform is technically not compliant with the statutory requirements imposed on criminal justice providers, and a court may choose to give less weight to their opinion or to admit it only with express acknowledgment of the quality-assurance gap. In the United States under Daubert, lack of accreditation is not in itself an exclusionary factor, but the absence of the quality-system features that accreditation requires, including validated methods and proficiency testing, may be explored under the error-rate and reliability factors. In India, non-accredited state FSL reports are routinely admitted, but courts have shown increasing willingness to ask about accreditation and to note its absence in judgments where the reliability of the opinion is in question.

1. Establish the laboratory's scope of accreditation
   Before accepting a case, confirm that the examination requested falls within the laboratory's NABL / ANAB / UKAS accredited scope. If the requested examination is outside scope, the examiner must either expand the scope (with the associated validation and assessment requirements) or decline and refer to an accredited provider.
2. Verify SOP version and equipment calibration
   Identify the current approved SOP for the examination to be conducted. Confirm that the equipment to be used is within its calibration interval. Record the SOP version number and equipment calibration dates in the case file at the start of the examination.
3. Conduct the examination per the validated SOP
   Follow the SOP exactly. Any departure from the SOP must be documented with a justification. If a material deviation from the validated SOP is required by the nature of the case material, it may be necessary to treat the result as outside the validated scope and qualify the conclusion accordingly.
4. Record all observations in contemporaneous notes
   Notes must be sufficient that another qualified examiner could understand what was done, what was observed, and how the conclusion was reached. ISO 17025 requires that the basis for results be traceable. Notes that reconstruct the examination after the conclusion was reached are not contemporaneous and will not survive disclosure scrutiny.
5. Blind verification by second examiner
   Arrange blind verification before the report is finalised. The verifier examines the same materials and documents their independent conclusion before being informed of the first examiner's conclusion. Document the verification process and outcome in the case file.
6. Quality review before report release
   Technical management reviews the report for compliance with the SOP, the conclusion scale, the report format required by court procedural rules, and the laboratory's quality management system. The report is not released until the technical review is complete and any corrections are made.