Frequency Estimates for Statistical Word Similarity Measures Egidio

Frequency Estimates for Statistical Word Similarity Measures Egidio Terra and C. L. A. Clarke School of Computer Science University of Waterloo Presenter: Cosmin Adrian Bejan

Introduction § A comparative study of two methods for estimating word cooccurence frequencies required by word similarity measures to solve human-oriented language tests. § Example of such tests: § determine the best synonym in a set of alternatives A={A 1, A 2, A 3, A 4} for a specific target word TW in a context C={w 1’, w 2’, … wn’} TW. § determine the best synonym when no context is available 2

Measuring Word Similarity § the notion for cooccurence of two words can be depicted by a contingency table: § each dimension represents a random discrete variable Wi with range A = {wi, wi}; § each cell represent the joint frequency where Nmax is the maximum number of cooccurences. 3

Similarity between two words Pointwise Mutual Information Χ 2 - test Likelihood ratio Average Mutual Information 4

Context supported similarity Cosine of Pointwise Mutual Information L 1 norm Contextual Average Mutual Information Contextual Jensen. Shanon Digergence Pointwise Mutual Information of Multiple words 5

Window-oriented approach § § § fw_i – frequency of wi fw_1, w_2 – cooccurence frequency of w 1 and w 2 N – size of the corpus in words P(wi) = fw_i/N fw_1, w_2 is estimated by the number of windows where the two words cooccur. § Nwt – number of windows of size t § P(w 1, w 2) = fw_1, w_2 / Nwt 6

Document-oriented approach § dfw_i – frequency of a word wi. It corresponds to the number of documents in which the words appears. § D – the number of documents § P(wi) = dfw_i/ D § dfw_1, w_2 – cooccurence frequency of two words – is the number of documents where the words cooccur. § P(w 1, w 2) = dfw_1, w_2 / D 7

Results for TOEFL test set 8

Results for TS 1 and context 9