Disputed Authorship in Literature and History

The Federalist Papers were published between October 1787 and May 1788 as a series of newspaper essays arguing for ratification of the U.S. Constitution. All eighty-five appeared under the pseudonym 'Publius'. The authors were identified as Hamilton, Madison, and Jay long before the statistical work, but the authorship of twelve specific papers was disputed between Hamilton and Madison for generations, based on conflicting lists left by both men.

Mosteller and Wallace approached the problem as a Bayesian inference exercise. They selected a set of common function words, computed the frequency distributions for each word across papers of known authorship, and used those distributions to calculate posterior probabilities for the disputed papers. Every one of the twelve disputed papers pointed to Madison, with odds in his favour ranging from very strong to overwhelming. The study was published in 1964 in the Journal of the American Statistical Association and has been replicated and extended by dozens of subsequent researchers.

What makes the Federalist Papers ideal as a test set is not just the historical documentation of authorship but the specific character of the task. Hamilton and Madison were both educated, prolific, contemporary authors writing in the same genre about the same subject. This controls for topic, period, and genre variation: any stylistic difference detected must reflect individual habit rather than subject matter or era. Few real-world attribution problems have such clean conditions.

No authorship question has generated more stylometric effort than whether the plays attributed to William Shakespeare were written by him. Numerous candidates have been proposed, including Francis Bacon, Christopher Marlowe, and the Earl of Oxford. Stylometric analysis has been brought to bear repeatedly, with results generally supporting Shakespeare's authorship of most of the canonical plays while finding credible evidence of co-authorship with John Fletcher in several late works.

The methodological difficulties are significant. The surviving Shakespeare corpus is small (approximately 900,000 words across the plays and poems), and all of it is in a single genre written over a short period. The comparison corpora for alternative candidates vary enormously in size and genre. The writing is often collaborative in ways that are impossible to disentangle cleanly. And the proposed non-Shakespearean candidates wrote in styles that do not match the plays nearly as well as Shakespeare's known style does, a result that consistently emerges from rigorous analysis.

In July 2013, the Sunday Times published a report identifying J.K. Rowling as the author of The Cuckoo's Calling, a crime novel published under the name Robert Galbraith. The identification was the work of forensic linguist Peter Millican, whose stylometric analysis found significant overlap between Galbraith's and Rowling's function-word profiles. Rowling confirmed the attribution the same day. The book immediately became a bestseller. The attribution was accurate, the method was valid, and Rowling's pseudonymity was destroyed within days of publication.

In 2016, the Italian journalist Claudio Gatti attributed the pseudonymous novelist Elena Ferrante to Anita Raja, a translator, using a combination of financial records and stylometric analysis. Ferrante had maintained her anonymity for over twenty years and had explained in interviews that her anonymity was central to how she wanted her work to be received. The attribution was contested by some stylometrists who argued the methodology was insufficiently rigorous, and Ferrante herself declined to confirm it.

The ethical debate turns on competing values that do not resolve neatly. One side: stylometric analysis is research; researchers have the right to publish valid findings; authorship is not a privacy-protected attribute in most legal systems. The other side: living authors who choose pseudonymity have reasons, and those reasons may include safety, creative freedom, and the desire to have their work judged on its own terms. Stripping that anonymity without consent harms real people. Both positions have serious advocates in the field, and no professional consensus has resolved the question.

A stylometric result that is publishable in a peer-reviewed linguistics journal may fall well short of what a court requires. The distinctions matter because the same methodology is deployed in both contexts and the gap between them is not always made explicit to the audiences consuming the results.

1. Academic publication standard
   A valid method, a reasonable corpus, replication across multiple features or methods, and peer review. Error rates are desirable but not always compulsory. Open-set validation is rare in literary stylometry; most studies are closed-set.
2. Expert witness report standard
   All of the above, plus: a known error rate specific to the case conditions (text length, candidate number, genre), interpretability for a lay jury, compliance with admissibility standards, and a statement of scope that distinguishes what the evidence can and cannot support.
3. Courtroom admissibility standard (Daubert, U.S.)
   The method must be testable, have a known error rate, be peer-reviewed, and be generally accepted in the relevant scientific community. The court applies these criteria at a pre-trial hearing; methods that fail are excluded before the jury hears them.

The practical implication for a forensic linguist is that stylometric evidence that would be accepted at a humanities conference may not survive a Daubert challenge. The converse is also true: a very conservative result that states only that the questioned text is 'more consistent with Candidate A than with the other candidates at a 91 per cent accuracy level' is both less dramatic and more defensible than a flat attribution claim.

The clearest way to think about this is to ask: what would it take to falsify the claim? For an academic attribution, the answer is a better-validated competing method that produces a different result. For a forensic attribution, the answer is a known error rate that bounds the probability of a false attribution, plus replication by at least one independent analyst, plus a comparison corpus that is large enough to have stable estimates. The burden is higher in court because the consequence of error is higher.

Several real forensic attributions have been rejected by courts precisely because the expert could not state a reliable error rate, had a comparison corpus that was too small, or could not explain the method in terms a jury could follow. These rejections are not failures of stylometry as a discipline; they are the correct functioning of admissibility standards. A method that cannot be explained or validated should not influence a criminal verdict, regardless of how impressive it looks to a specialist audience.