Review of Some Concepts from the IIR Textbook

Review of Some Concepts from the IIR Textbook IST 516 Fall 2010 Dongwon Lee, Ph. D.

Mean Average Precision (Ch 8. 4) l Def: l l l AP (Average Precision): for a single query, average of the precisions obtained for the set of top k documents existing after each relevant document is retrieved MAP: average of all APs for all queries Formula: l MAP (Q) = 1/|Q| SUM (MAP for each query) 2

MAP Example 1/3 l An IR system returns the following top-5 ranked answers for a given query q 1: l l l Rank Relevant? 1 Yes 2 No 3 Yes 4 Yes 5 No Rank 1: P = 1/1 = 1 Rank 3: P = 2/3 = 0. 67 Rank 4: P = 3/4 = 0. 75 MAP(q 1) = 1/3 (1+0. 67+0. 75) = 0. 87 3

MAP Example 2/3 l An IR system returns the following top-5 ranked answers for a given query q 2: l l Rank Relevant? 1 No 2 Yes 3 No 4 No 5 Yes Rank 2: P = 1/2 = 0. 5 Rank 5: P = 2/5 = 0. 4 MAP(q 2) = 1/2 (0. 5+0. 4) = 0. 45 4

MAP Example 3/3 l l MAP(Q), where Q={q 1, q 2} MAP(Q) l l l = ½ (MAP(q 1) + MAP(q 2)) = ½ (0. 87 + 0. 45) = 0. 66 5

Centroid (Ch 16. 4) l Def: Center of a mass of a set of points (or vectors or documents) l l l Center of gravity D: a set of points d: a point in D 6

Centroid Example l 2 dimensional space l l p 1: (5, 10) p 2: (10, 5) p 3: (10, 15) Centroid of p 1, p 2, and p 3 l 1/3 (5+10+10, 10+5+15) = (25/3, 30/3) 7

Rocchio Algorithm (Ch 9. 1. 1) § § § Dr = set of known relevant doc vectors Dnr = set of known irrelevant doc vectors qm = modified query vector; q 0 = original query vector α, β, γ: weights (hand-chosen or set empirically) New query moves toward relevant documents and away from irrelevant documents Centroid of a set of known relevant documents Centroid of a set of unknown relevant documents 8

Rocchio Example 1/5 Initial query x x x o o x x x x o x x x known non-relevant documents o known relevant documents 9

Rocchio Example 2/5 Initial query x x x o o x x x x o x x x known non-relevant documents o known relevant documents 10

Rocchio Example 3/5 Initial query x x x o o x x x x o x x x known non-relevant documents o known relevant documents 11

Rocchio Example 4/5 Initial query x x x o o x x x x o x x x known non-relevant documents o known relevant documents 12

Rocchio Example 5/5 Initial query x x x o o x x x x o x x Revised query x known non-relevant documents o known relevant documents 13