Text Categorization Moshe Koppel Lecture 3 Authorship Attribution

Text Categorization Moshe Koppel Lecture 3: Authorship Attribution Mostly my own stuff together with Jonathan Schler, Shlomo Argamon, Ido Dagan, Jamie Pennebaker, Jonathan Fine, Kfir Zigdon, Iris Zigdon, Navot Akiva, Dror Mughaz, Eran Messeri

The Authorship Attribution Problem Given the known writings of a small number of authors A 1, …, An , determine which of them wrote document X. The vanilla version is actually not very difficult.

History of Authorship Attribution • Researchers have been trying to solve this problem since the 19 th century • Early research focused on classical works: Bible, Shakespeare, etc. • The most famous work is Mosteller & Wallace (1964) on the Federalist Papers • The proliferation of online text offers many new research opportunities

Authorship – Realistic Versions • No suspects – Can we profile the author of X? • One suspect – Can we ascertain guilt or innocence? • Thousands of suspects – Can we pick a needle from the haystack?

Text Categorization

Features for Authorship Attribution What features do we want? • Used consistently by given author regardless of document type Ø Eliminates topic words • Might be used differently by different authors Ø Our challenge: identify these

Olden Days • Early researchers sought a single “magic” feature • Mainly, they looked at complexity features: – Sentence or word length – Type/token ratio – Hapax legomena • The idea was the magic feature had a distinct constant value for each author • This approach isn’t taken seriously these days.

Features for Categorizing by Style • Function words (and, of, the, . . ) [Mosteller-Wallace 64]

Features for Categorizing by Style • Function words • Syntax – POS n-grams

Features for Categorizing by Style • Function words • Syntax – POS n-grams • SFL trees

SFL Trees Conjunctions Conj. Extension and, or, but, yet, however, … Conj. Elaboration for_example, indeed, … Conj. Enhancement Conj. Spatiotemporal then, beforehand, afterwards, while, during, … Conj. Causal. Conditional if, because, since, hence, thus, therefore, … • Similar trees for prepositions, pronouns, modal verbs • Covers all function words, some POS • Doesn’t require POS tagging

Features for Categorizing by Style • • Function words Syntax SFL trees Morphology (grammatical prefixes and suffixes, e. g. –ing, re-)

Features for Categorizing by Style • • • Function words Syntax SFL trees Morphology Complexity measures (e. g. avg. word length, avg. sentence length, various entropy measures)

Features for Categorizing by Style • • • Function words Syntax SFL trees Morphology Complexity measures “Unstable” words (features that might be replaced in a rewrite, e. g. huge: large)

Measuring Instability An interesting idea: Translate many documents to another language and translate back. See what sticks. We did just that on Reuters 21578 and BNC*. * We used five different languages to ensure robustness.

Word Stability -- Examples huge ratio device onto has over nine September . 0. 06. 28. 55. 98. 99

Features for Categorizing by Style • • Function words Syntax SFL trees Morphology Complexity measures “Unstable” words Idiosyncrasies

Idiosyncrasies Human attribution experts use idiosyncrasies e. g. spelling errors, neologisms, quirky syntax • Use spelling/grammar checker to identify “errors” • Categorize errors by type e. g. ie ei, nn n

Some problematic feature sets • Many researchers use content words • A really easy feature set is character ngrams • These sets are problematic since they ought to fail when an author changes topics • In practice, they work very well

Learning -- Historical • From 1964 until 1990’s all work had flavor of Naïve Bayes • Idea was to characterize all of each candidate author’s work and see which one X is most similar to • Various proximity measures used (often after dimension reduction) • Since 1990’s variety of learning methods applied

Learning Algorithms Let’s try some old friends (and one new one): • Naïve Bayes • Decision Trees • Winnow • SVM • Bayesian regression

Three testbed corpora • Emails between two correspondents (date split) – Some as short as one word • Two books by each of nine authors (book split) – Different books have different content • Twenty bloggers (date split) – Lots of authors

Email corpus – 2 authors

Literature corpus – 9 authors

Blog corpus – 20 authors

Some Lessons Some lessons: • SVM and Bayesian regression are the best learners. • FW and single POS are generally enough. • In fact, SFL trees alone are often enough. • Content words and character n-grams work surprisingly well Some other facts worth knowing: • Common unstable words are better than FW. • For unedited texts, idiosyncrasies are best. • For some languages, morphology is needed.