N-Gram
Definition
A contiguous sequence of n items (characters, words, or part-of-speech tags) extracted from text. Character n-grams and word n-grams are both standard attribution features.
- Definition
- Contiguous sequence of n items from text
- Item types
- Characters, words, or part-of-speech tags
- Field use
- Standard feature in authorship attribution
- Common sizes
- Character n-grams often n=2 to 4, word n-grams often n=1 to 3
Common questions
Why are character n-grams often preferred over word n-grams in authorship attribution?+
Character n-grams capture spelling habits, punctuation spacing, and morphological patterns that persist even in short or noisy texts such as messages, and they hold up better against typos and out-of-vocabulary words than word-level features. Word n-grams instead capture phrasing and vocabulary choice, so many systems combine both.
Does a larger n always give more reliable attribution?+
No. Larger n captures more context but each specific n-gram becomes rarer, so counts become sparse and unreliable on short documents, which can hurt rather than help classification. Practitioners typically test a range of n values and pick what performs best on validation data for the text length involved.
How do n-gram frequencies get turned into an attribution decision?+
Frequencies of many n-grams are compiled into a feature vector for each candidate author and the questioned document, then a statistical or machine-learning classifier compares the questioned document's vector against known-author profiles to estimate the most likely author or to flag stylistic inconsistency.
Related terms
- Closed-Set Attribution
- An attribution task where the true author is assumed to be one of a defined list of candidates. The system ranks candidates;...
- Feature Extraction
- The process of converting raw text into a numerical vector of linguistic measurements. The choice of features determines what signal the classifier...
- Function Words
- Grammatical words, prepositions, conjunctions, articles, pronouns, with little independent content meaning but high frequency in any text. Because they are used without...
- Idiolect
- The language variety specific to an individual, comprising their characteristic vocabulary, syntactic preferences, spelling habits, punctuation patterns, and discourse-level style. Authorship attribution...
- Open-Set Attribution
- An attribution task where the true author may or may not appear in the candidate pool. The system must both rank candidates...