Stylometry and Statistical Distance Methods

John Burrows introduced Delta in a 2002 methodological paper, 'Delta: A Measure of Stylistic Difference and a Guide to Likely Authorship', published in Literary and Linguistic Computing. The algorithm is transparent enough to explain in a few sentences. Take a large reference corpus and extract the n most frequent words (typically 100-500). For each word, compute its mean frequency and standard deviation across the corpus. Standardise each text's word frequencies by subtracting the mean and dividing by the standard deviation, producing a z-score. Delta is then the average absolute difference in z-scores across the feature words, between the questioned text and each candidate. The candidate with the smallest Delta is the closest stylistic match.

The z-score standardisation is what gives Delta its stability. Without it, common words dominate simply because they are common. After standardisation, a word used at twice the corpus average contributes the same to the distance as a rare word used at twice its own average. Each feature contributes to the distance in proportion to its deviation from the corpus norm, regardless of its raw frequency.

No single variant dominates across all conditions. Published comparative studies suggest Cosine Delta performs best on short texts and Eder's Delta handles non-Gaussian distributions well. The practical advice for forensic work is to run multiple variants and treat strong cross-variant agreement as a sign of a reliable result. Where variants disagree, that disagreement is evidence to report, not to hide.

Delta scores are numbers. PCA turns them into a picture. Principal component analysis takes the high-dimensional feature vectors for a set of texts and projects them onto two or three axes that capture the most variance in the data. Texts by the same author cluster together; texts by different authors sit apart. A plot of the first two principal components is standard in published stylometric studies because it shows the authorship structure of a corpus at a glance.

Cluster analysis is a complementary approach. Hierarchical clustering groups texts by similarity and displays the result as a dendrogram, a tree where closely related texts branch off together. Both PCA plots and dendrograms are valuable in court because they make the distance structure visible to a non-specialist audience. They do not, by themselves, assign authorship; they show where the questioned document sits relative to candidates in a feature space the expert has constructed.

A single Delta score for a document treats the entire text as a uniform block. Rolling Delta relaxes that assumption. The method slides a window of fixed size (commonly 1,000-5,000 words) across the text and computes the Delta to each candidate at each window position. The output is a graph: the x-axis is the position in the document; the y-axis is the Delta distance to each candidate. Where one candidate's line drops while another's rises, the method is flagging a stylistic shift.

This is useful for two forensic scenarios. The first is collaborative authorship: a document attributed to one person may turn out to show a clear stylistic boundary mid-way through, consistent with a second author drafting the later sections. The second is revision or ghost-writing: a base text written by one person and substantially revised by another can show a distinctive layering of styles that a single global score would obscure. In literary scholarship, rolling Delta was used to probe the joint authorship of plays attributed to Shakespeare and Fletcher.

Support vector machines (SVMs), random forests, and neural networks consistently outperform Delta methods on laboratory benchmarks, often by a wide margin on large candidate sets. The question for forensic application is not accuracy in the abstract; it is whether the accuracy estimate is reliable for the specific case conditions (text length, number of candidates, genre, language), and whether the method is interpretable enough for a court to scrutinise.

The admissibility debate is live in multiple jurisdictions. U.S. federal courts applying the Daubert standard require that a method be testable, have a known error rate, be peer-reviewed, and be generally accepted in the relevant scientific community. Delta satisfies these criteria better than most neural approaches. UK courts under the reliability standard established in R v Dlugosz [2013] EWCA Crim 2 similarly weigh method transparency heavily. The practical position most forensic linguists adopt is: use machine-learning methods to form an initial view, then confirm with a Delta-based method that can be explained transparently in the report and withstand cross-examination.

Stylometric methods can produce impressive numbers in controlled settings. The question is whether those numbers are honest estimates of performance on the kind of text the forensic case involves. Two practices undermine honest validation more than any others: over-fitting to a training corpus, and validating on data that overlaps with training data.

The closed-corpus trap is specifically the problem of validating a method only on texts from the same set of authors used to build the feature model. In that setting, the method has, in effect, seen the answer before the test. Reliable validation requires held-out authors, texts written at different times or in different registers, and ideally comparison against a population of texts from writers not in the candidate set, to establish the rate at which the method assigns a questioned text to the wrong person when the true author is absent.

1. k-fold cross-validation
   Divide the labelled corpus into k equal folds. Train on k-1 folds and test on the held-out fold. Repeat k times and average the error rates. For forensic purposes, k=10 is a common choice.
2. Leave-one-out cross-validation
   A special case where k equals the number of texts. Each text is tested exactly once as a held-out item. Maximises the training data used but is computationally expensive on large corpora.
3. Imposter method
   A specific open-set validation technique that introduces 'impostor' texts from outside the candidate set. The method must reject these as not matching any candidate. This directly tests the open-set problem that real forensic cases face.