Recuperao de Informao B Cap 02 Modeling Set

Recuperação de Informação B Cap. 02: Modeling (Set Theoretic Models) 2. 6 September 08, 1999

Set Theoretic Models n n n The Boolean model imposes a binary criterion for deciding relevance The question of how to extend the Boolean model to accomodate partial matching and a ranking has attracted considerable attention in the past We discuss now two set theoretic models for this: u Fuzzy Set Model u Extended Boolean Model

Fuzzy Set Model n n Queries and docs represented by sets of index terms: matching is approximate from the start This vagueness can be modeled using a fuzzy framework, as follows: u with each term is associated a fuzzy set u each doc has a degree of membership in this fuzzy set n n This interpretation provides the foundation for many models for IR based on fuzzy theory In here, we discuss the model proposed by Ogawa, Morita, and Kobayashi (1991)

Fuzzy Set Theory n n Framework for representing classes whose boundaries are not well defined Key idea is to introduce the notion of a degree of membership associated with the elements of a set This degree of membership varies from 0 to 1 and allows modeling the notion of marginal membership Thus, membership is now a gradual notion, contrary to the crispy notion enforced by classic Boolean logic

Fuzzy Set Theory n Definition u. A fuzzy subset A of U is characterized by a membership function (A, u) : U [0, 1] which associates with each element u of U a number (u) in the interval [0, 1] n Definition u Let A and B be two fuzzy subsets of U. Also, let ¬A be the complement of A. Then, F F F (¬A, u) = 1 - (A, u) (A B, u) = max( (A, u), (B, u)) (A B, u) = min( (A, u), (B, u))

Fuzzy Information Retrieval n n Fuzzy sets are modeled based on a thesaurus This thesaurus is built as follows: u Let vec(c) be a term-term correlation matrix u Let c(i, l) be a normalized correlation factor for (ki, kl): c(i, l) = n(i, l) ni + nl - n(i, l) • ni: number of docs which contain ki • nl: number of docs which contain kl • n(i, l): number of docs which contain both ki and kl n We now have the notion of proximity among index terms.

Fuzzy Information Retrieval n The correlation factor c(i, l) can be used to define fuzzy set membership for a document dj as follows: (i, j) = 1 - ki dj (1 - c(i, l)) • (i, j) : membership of doc dj in fuzzy subset associated with ki n n The above expression computes an algebraic sum over all terms in the doc dj A doc dj belongs to the fuzzy set for ki, if its own terms are associated with ki

Fuzzy Information Retrieval (i, j) = 1 - n ki dj (1 - c(i, l)) • (i, j) : membership of doc dj in fuzzy subset associated with ki n If doc dj contains a term kl which is closely related to ki, we have u c(i, l) ~1 u (i, j) ~ 1 u index ki is a good fuzzy index for doc

Fuzzy IR: An Example Ka cc 3 n n n Kb cc 2 cc 1 q = ka (kb kc) vec(qdnf) = (1, 1, 1) + (1, 1, 0) + (1, 0, 0) = vec(cc 1) + vec(cc 2) + vec(cc 3) (q, dj) = (cc 1+cc 2+cc 3, j) = 1 - (a, j) (b, j) (c, j)) * (1 - (a, j) (b, j) (1 - (c, j))) * (1 - (a, j) (1 - (b, j)) (1 - (c, j))) Kc

Fuzzy Information Retrieval n n n Fuzzy IR models have been discussed mainly in the literature associated with fuzzy theory Experiments with standard test collections are not available Difficult to compare at this time

Extended Boolean Model n n n Booelan retrieval is simple and elegant But, no ranking is provided How to extend the model? u interpret conjunctions and disjunctions in terms of Euclidean distances