Feature Extraction
Definition
The process of converting raw text into a numerical vector of linguistic measurements. The choice of features determines what signal the classifier can see, and is as consequential as the classifier itself.
- Field
- Authorship attribution / stylometry
- Input
- Raw text
- Output
- Numerical vector of linguistic measurements
Common questions
What kinds of features are typically extracted in stylometric authorship work?+
Common feature types include function word frequencies, sentence length distributions, character n-grams, punctuation usage and vocabulary richness measures. Each captures a different aspect of writing style, and combining several usually outperforms relying on any single feature type.
Why does the choice of features matter as much as the classifier algorithm that follows it?+
A classifier can only detect patterns present in the features it is given. Features tied to topic or genre rather than style can make two different authors on the same subject look similar, so poor feature choice caps accuracy regardless of how sophisticated the downstream classifier is.
Related terms
- Closed-Set Attribution
- An attribution task where the true author is assumed to be one of a defined list of candidates. The system ranks candidates;...
- Function Words
- Grammatical words, prepositions, conjunctions, articles, pronouns, with little independent content meaning but high frequency in any text. Because they are used without...
- Idiolect
- The language variety specific to an individual, comprising their characteristic vocabulary, syntactic preferences, spelling habits, punctuation patterns, and discourse-level style. Authorship attribution...
- N-Gram
- A contiguous sequence of n items (characters, words, or part-of-speech tags) extracted from text. Character n-grams and word n-grams are both standard...
- Open-Set Attribution
- An attribution task where the true author may or may not appear in the candidate pool. The system must both rank candidates...