Perplexity
Definition
A measure of how surprising a sequence of words is to a language model. LLMs tend to generate low-perplexity text (predictable word choices); human writing often shows higher and more variable perplexity. Detection tools exploit this, but the distinction breaks down with formal academic prose.
- Metric type
- Statistical predictability measure over a token sequence
- Low score
- Predictable word choices, typical of LLM output
- High score
- More variable word choices, typical of human writing
- Known limitation
- Formal academic prose can score low even when human-written
Common questions
Can a low perplexity score alone prove a document was AI-generated?+
No. Perplexity is one signal among several that detection tools combine, and courts have rejected it as standalone proof because formal, templated or heavily edited human writing can also score low.
Why does perplexity-based detection struggle with academic or legal writing?+
Those genres already favour conventional phrasing and predictable sentence structure, so the perplexity gap between human and machine text narrows or disappears, producing false positives against skilled human writers.
Related terms
- AI Detection Classifier
- A machine-learning system trained to discriminate between human-written and LLM-generated text by measuring features such as perplexity, burstiness, and n-gram probabilities. Current...
- Burstiness
- The variance in sentence-level complexity across a passage. Human writing tends to mix complex and simple sentences unevenly; LLM output tends to...
- Human-LLM Collaboration
- Text production in which a human contributes prompts, editing decisions, and intentional content while an LLM generates or transforms the prose. The...
- Large Language Model (LLM)
- A neural network trained on large text corpora to predict likely next tokens. At inference, it generates text by sampling from probability...
- Stylometric Signature
- The set of measurable linguistic features : function-word frequencies, sentence length, punctuation patterns, vocabulary richness : that characterises a specific writer's output...