Computer-Assisted Handwriting Analysis: FISH, WANDA, CEDAR

FISH (Forensic Information System for Handwriting) was developed at the Bundeskriminalamt (BKA) in Wiesbaden, Germany, beginning in the late 1990s, with a version operational by 2003. Its primary function is database search: given a questioned handwriting sample, FISH returns a ranked list of candidate writers from the database of known writing samples. It is a triage and investigative tool, not an identification tool; the ranked list is reviewed and ranked candidates are then subjected to traditional human examination.

The feature extraction in FISH is based on the theta-features approach, in which the distribution of stroke directions (the angle of each short stroke segment relative to horizontal) is computed as a histogram. This direction-distribution histogram, often called a theta histogram, is compact to compute and relatively robust to changes in writing size, because direction distributions are largely scale-invariant. FISH computes theta histograms at the word level and at the global-document level, then uses a distance metric to compare the questioned writing against the database entries.

Validation studies conducted by the BKA and published in peer-reviewed journals (Franke, Schomaker and colleagues) reported rank-1 retrieval rates (correct writer in the first result returned) of approximately 30 to 40 per cent on their internal German-writer database, rising to approximately 80 per cent at rank 10 (correct writer in the top 10 returned). These figures are meaningful for investigative triage: if the correct writer is in the database, FISH returns them in the top 10 results 80 per cent of the time, substantially narrowing the comparison workload for human examiners. FISH has been used by BKA examiners in terrorism-related anonymous correspondence cases and in fraud casework involving large volumes of handwritten documents.

The limitation of theta-feature approaches is their sensitivity to handwriting style: the direction distributions of a highly cursive writer and a print writer are different in ways that can dominate the comparison and make cross-style comparisons unreliable. FISH performs best when both the questioned sample and the database entries are in broadly similar styles (e.g. German cursive, or German print). Its performance degrades when the questioned writing is in a style under-represented in the database, or when the writing is heavily disguised.

FISH has not been independently validated by laboratories outside the BKA network using its operational database, which is a limitation that the PCAST report would later flag as a generic weakness of CAHA systems: system-level validation by the developer, without independent replication on a comparable operational database, does not meet the standard required for court-level reliability demonstration.

CEDAR-FOX (Computer for Expert Document Analysis and Recognition) was developed at the Center for Unified Biometrics and Sensors (CUBS), University at Buffalo (SUNY), by Sargur Srihari and colleagues, with foundational publications in 2002 (Srihari, Cha, Arora, Lee, IEEE Transactions on Pattern Analysis and Machine Intelligence) and subsequent extensions through the 2000s and early 2010s. The work was funded substantially by US Department of Justice grants and was explicitly framed in the context of providing Daubert-compliant reliability data for handwriting examination.

The feature set in CEDAR-FOX is richer than FISH's theta features. Srihari's team extracted micro-features (stroke curvature, pen pressure derived from gray-level intensity in scanner images, pen-lift frequency), macro-features (word-level slant, letter spacing, aspect ratios of letters), and allographic features (the specific visual form of each letter, scored across a set of learned categories). The combination of these features was processed through support vector machines (SVM) for writer verification: given a questioned document and a candidate writer's exemplars, the SVM produces a similarity score.

The 2002 paper reported accuracy results on a dataset of 1,500 writers (the CEDAR Letter dataset, available to researchers), achieving writer-identification error rates in the range of 6 to 8 per cent for same-session comparisons and higher error rates for cross-session comparisons (where the writing style may vary more between the questioned and exemplar samples). These were the first rigorously designed and independently reproducible error-rate estimates for a CAHA system published in a major peer-reviewed venue, and they were widely cited in subsequent discussions of handwriting examination reliability.

CEDAR-FOX has been used in expert testimony in US federal courts, including testimony by Srihari himself. The admissibility of CEDAR-FOX testimony has been contested under Daubert in several proceedings, with courts generally admitting it on the basis of its documented error rate while often limiting the weight assigned to it. The conclusion scales and courtroom language used to translate these scores into testimony are covered separately. The technique is not in routine operational use by the FBI or by most accredited forensic laboratories; it has functioned primarily as a research demonstration and as a model for what validated CAHA should look like, rather than as an operational tool.

WANDA (Writer Automatic Non-parametric Database Approach) was developed within the European FIDIS (Future of Identity in the Information Society) network research project, with contributions from research groups in the Netherlands (Nijmegen Institute for Cognition and Information, Radboud University Nijmegen, and the Netherlands Forensic Institute), Germany, and Switzerland. WANDA uses a non-parametric, nearest-neighbour approach: the questioned writing is characterised by a feature vector (combining direction features, pressure proxies, and texture features), and the system returns the nearest neighbours from the reference database. The "non-parametric" designation means the system does not assume a parametric distribution for writer features, which is appropriate given the diversity of writing styles across a population database.

WANDA was evaluated in published studies (Schomaker, Bulacu, and colleagues, 2004-2012) on the ICDAR2011 WriterIdentification database (a public benchmark dataset of handwritten text from 650 writers), achieving top-1 retrieval rates competitive with FISH and CEDAR-FOX on that dataset. However, the ICDAR dataset uses controlled text-copying tasks, not naturalistic casework samples, which limits the generalisability of benchmark results to operational conditions where writing quality, sample size, and disguise factors vary.

FLASH-ID (Forensic Linguistic and Handwriting Identification System) is a commercial product developed by a UK-based company, used by several European police and security services primarily for batch handwriting comparison in fraud and document-crime investigations involving large volumes of handwritten material. Its computational basis is proprietary, limiting independent validation. ENFSI's external quality assurance programme has included FLASH-ID in comparison exercises; results from those exercises are available to ENFSI member laboratories but not published in the open literature.

The European admissibility environment differs from the US Daubert standard. German courts (under the Bundesgerichtshof framework) admit scientific evidence if the method is recognised in the relevant scientific community, without a gatekeeper hearing of the US type. French courts apply the intime conviction standard, giving judges wide discretion over the weight assigned to technical evidence. In England and Wales, the Criminal Procedure Rules Part 19 and the R v. Pabon (2018) and R v. Henderson (2010) decisions have tightened requirements for expert evidence, moving toward a de facto reliability check closer to the US position. Under all European frameworks, the absence of a validated error rate for a CAHA system is not automatically fatal to admissibility, but it weakens the weight the court assigns.

The application of deep learning to writer verification has accelerated sharply since 2015, following the broader success of convolutional neural networks (CNNs) in visual pattern recognition. The writer-verification task is typically framed as metric learning: train a neural network to map handwriting patches to a feature space where patches from the same writer are close together and patches from different writers are far apart, then use a threshold on the distance metric to make verification decisions.

Bhunia et al. (2021) and related work in the handwriting domain explored few-shot writer adaptation. He et al. (2022, "Writer Identification Using Handwritten Word Images") applied transformer-based vision models to the IAM Handwriting database (a widely used academic benchmark of English handwriting from 657 writers) and reported verification equal-error rates below 5 per cent on that benchmark. Earlier work by Christlein, Bernecker, Maier, and Angelopoulou (2017, "Writer Identification Using GMM Supervectors and Exemplar-SVMs") achieved comparable performance on the ICDAR2013 and CVL datasets using a combination of GMM supervectors and exemplar-SVM re-scoring.

The PCAST report's critique applies directly to these results: benchmark performance on curated, controlled handwriting datasets does not constitute validation for forensic use. The IAM database consists of writers copying printed text in a single session; forensic casework samples come from naturalistic writing, vary in quantity and quality, may be disguised, and must be compared across time gaps of months or years during which a writer's style may have evolved. No controlled study with operational forensic samples and verified ground truth has been published demonstrating that deep-learning writer verification matches its benchmark performance in a forensic context.

The machine learning research community and the forensic science community have partially different definitions of "validation." The cognitive bias research by Dror 2006 is equally relevant to human-in-the-loop CAHA workflows, where the human reviewer sees the system score before completing their own assessment. In machine learning, a system is validated against a held-out test set from the same distribution as the training set. In forensic science, validation requires that the system be tested against samples from the operational distribution (naturalistic questioned documents, not curated copies), with a statistically appropriate sample size, by examiners or operators who are blind to ground truth, and with a reported false-positive rate that is conservative for court purposes. These two definitions intersect but are not the same, and the gap between them is what the PCAST critique addressed.

The President's Council of Advisors on Science and Technology (PCAST) report "Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods" (September 2016) was a landmark document for multiple forensic disciplines, including fingerprint examination, bite marks, hair analysis, firearm toolmarks, and handwriting. For handwriting, PCAST reviewed the existing empirical literature and applied two evaluative criteria: foundational validity (has the method been shown to be repeatable, reproducible, and accurate in the relevant context?) and validity as applied (do the specific laboratories and examiners applying the method meet the performance standards demonstrated in foundational research?).

PCAST found the existing studies on handwriting examination methodologically inadequate for demonstrating foundational validity. The problems identified were: small sample sizes (most studies used fewer than 200 writers and fewer than 500 questioned-exemplar pairs); absence of population-representative samples (most studies used convenience samples of student writers rather than operational forensic samples); absence of reported false-positive rates (most studies reported identification accuracy without separately reporting the rate at which a different writer was incorrectly identified as the writer); and absence of blind comparison conditions (in some studies, examiners knew which questioned samples were genuine matches, defeating the purpose of measuring error rates).

PCAST recommended that the forensic document examination community undertake well-designed black-box studies: studies in which a large population of qualified examiners, in accredited laboratories, compare a large set of questioned writing samples against exemplar sets under blind conditions (examiners do not know which comparisons are same-writer and which are different-writer), with reported sensitivity (true-positive rate), specificity (true-negative rate), false-positive rate, and false-negative rate separately documented.

The forensic document examination community, through the American Board of Forensic Document Examiners (ABFDE) and the American Society of Questioned Document Examiners (ASQDE), contested the PCAST findings on several grounds: that the studies reviewed did not represent the full empirical literature, that practitioner proficiency testing (which exists and shows generally high accuracy rates) was relevant to validity as applied, and that the PCAST criterion for foundational validity was set higher than that applied to pattern-evidence disciplines with longer admissibility history (fingerprints). The debate is documented in several peer-reviewed responses published in the Journal of Forensic Sciences and Forensic Science International between 2016 and 2020.

In US courts, PCAST's influence has been partial. Some federal district courts have used the PCAST findings to limit or exclude handwriting testimony in specific cases (US v. Chavez, 2018; US v. Prime, 2002, predates PCAST but shares the admissibility-challenge logic). Other courts have admitted handwriting testimony while limiting it to the examiner's observations (what features were found) rather than to conclusions (this was written by X). In England, the Criminal Procedure Rules Part 19 and the parallel expert evidence guidelines under the Civil Procedure Rules have not incorporated the PCAST framework explicitly, but the post-2016 English cases on expert admissibility have increasingly required experts to document their methodology and its empirical basis.

The National Institute of Standards and Technology (NIST) published "Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach" in 2020 (NISTIR 8282). This report, produced collaboratively with the American Academy of Forensic Sciences (AAFS) and the forensic document examination community, built on the PCAST critique to produce a specific research and improvement agenda. It is the most comprehensive policy document on handwriting examination produced to date and is referenced by the OSAC Forensic Document Examination Subcommittee in its current standard-revision work.

NIST 2020 identified six priority research areas: (1) large-scale black-box studies of human examiner accuracy across the full nine-point conclusion scale; (2) proficiency testing with documented performance metrics and reporting; (3) computational tools for feature extraction and comparison that produce validated similarity scores with uncertainty bounds; (4) database development (especially for non-Latin scripts); (5) workflow and reporting standards that separate observations (what was found) from conclusions (what it means) in the formal examination report; and (6) training and certification standards aligned with the performance benchmarks established by research.

Progress has been uneven. On the black-box study front, the Forensic Science International study by Caligiuri, Mohammed, and Motta (2020) is one of the few post-PCAST studies that approximated the recommended design, using 109 examiners on 4,500 questioned-exemplar pairs with blind ground truth. It found false-positive rates (same-writer incorrectly concluded as different-writer or inconclusive) and false-negative rates that were generally comparable to prior studies, with overall conclusion accuracy above 80 per cent but with substantial variability across examiners and across difficulty strata.

On the computational tools front, the post-2020 deep-learning writer-verification literature (He 2022, and subsequent work using vision transformers by Peer, Bhunia, and colleagues) has produced increasingly competitive benchmark performance on public datasets, but the gap between benchmark validation and forensic validation (documented above) has not closed. The OSAC Forensic Document Examination Subcommittee has published a roadmap for integrating computational tools into forensic workflows as decision-support systems, not as autonomous identification engines, with human review mandatory at every decision stage.

The Daubert standing of CAHA systems in US courts in 2026 is provisional. No CAHA system has been admitted as a primary identification tool without accompanying human expert testimony. The systems are generally admitted as investigative aids or as supporting evidence when the human examiner also testifies. In Germany, FISH is used operationally but does not testify directly; BKA examiners testify based on the traditional examination, with FISH as a documented investigation step. In England, FLASH-ID has been used in document-crime investigations but similarly supports rather than replaces human examination in court.

1. Query formulation
   Define the forensic question (writer identification, verification, or database triage) and the available material (questioned document, exemplar set, database). CAHA system selection depends on the question type and available system.
2. Sample preparation and digitisation
   Scan the questioned document and exemplars at consistent resolution (typically 600-1200 dpi) with calibrated illumination. Document scanner settings and image format in the case record.
3. Feature extraction
   Run the CAHA system's feature extraction pipeline on both questioned and exemplar images. Document the system version, parameters, and any pre-processing steps applied.
4. Similarity computation
   Compute the system's similarity score or ranked retrieval list. Document the raw output including score distributions if available.
5. Threshold application
   Apply validated decision thresholds if the system provides them, or report the score without a threshold decision if the system's thresholds have not been validated for the operational context.
6. Human expert review
   Conduct traditional handwriting comparison of the questioned document against exemplars. The CAHA output is one input to this review, not a substitute for it. Document which features were assessed, the comparison methodology, and the conclusion on the nine-point scale.
7. Report preparation
   Report observations and conclusions separately. Document the role of the CAHA system (triage, support, investigative aid) and its limitations explicitly. State the system's validation status and any applicable validation gap.