Outline Introduction Ranking models Selecting ranking techniques Data

Outline ¡ Introduction ¡ Ranking models ¡ Selecting ranking techniques ¡ Data structures and algorithms ¡ The creation of an inverted file ¡ Searching the inverted file ¡ Stemmed and unstemmed query terms ¡ A Boolean systems with ranking ¡ Pruning 2

Introduction ¡ Boolean systems Providing powerful on-line search capabilities for librarians and other trained intermediaries Providing very poor service for end-users who use the system infrequently ¡ The ranking approach Inputting a natural language query without Boolean syntax Producing a list of ranked records that “answer” the query More oriented toward end-users 3

Introduction (cont. ) ¡ Natural language/ranking approach is more effective for end-users The results being ranked based on cooccurrence of query terms modified by statistical term-weighting eliminating the often-wrong Boolean syntax used by end-users providing some results even if a query term is incorrect 4

Figure 14. 1 Statistical ranking Term Factors Qry. Vtr. Information 1 1 Retrieval systems 0 1 1 0 1 0 0 1 Human, factors, help, systems 1 0 Rec 3. Vtr. Operation Human, factors, information, retrieval Rec 2. Vtr. Human factors in information retrieval systems Rec 1. Vtr. Help 1 1 0 Factors, operation, systems 1 0 0 0 1 5

Figure 14. 1 Statistical ranking ¡ Simple Match Query (1 1 0 1 1) Rec 1 (1 1 0 1 0 1 0) = 4 ¡ Weighted Match Query (1 1 0 1 1) Rec 1 (2 3 0 5 0 3 0) = 13 Query (1 1 0 1 0 1 1) Rec 2 (1 0 1 1 0 0 1) Rec 2 (2 0 4 5 0 0 1) (1 0 0 1) = 3 (2 0 0 5 0 0 1) = 8 Query (1 1 0 1 0 1 1) Rec 3 (1 0 0 0 1) Rec 3 (2 0 0 0 2 0 1) (1 0 0 0 1) = 2 (2 0 0 0 1) = 3 6

Ranking models ¡ Two types of ranking models ranking the query against Individual documents ¡Vector space model ¡Probabilistic model ranking the query against entire sets of related documents 7

Ranking models (cont. ) ¡ Vector space model Using cosine correlation to compute similarity Early experiments ¡SMART system (overlap similarity function) Results ¡Within document frequency weighting > no term weighting ¡Cosine correlation with frequency term weighting > overlap similarity function ¡Salton & Yang (1973) (Relying on term importance within an entire collection) Results ¡Significant performance improvement using the withindocument frequency weighting + the inverted document frequency (IDF) 8

Ranking models (cont. ) ¡ Probabilistic model Terms appearing in previously retrieved relevant documents was given a higher weight Croft and Harper (1979) ¡Probabilistic indexing without any relevance information ¡Assuming all query terms have equal probability ¡Deriving a term-weighting formula 9

Ranking models (cont. ) ¡ Probabilistic model Croft (1983) ¡Incorporating within-document frequency weights ¡Using a tuning factor K ¡Result Significant improvement over both the IDF weighting alone and the combination weighting 10

Other experiments involving ranking ¡ Direct comparison of similarity measures and term- weighting schemes 4 types of term frequency weightings (Sparch Jones, 1973) ¡Term frequency within a document ¡Term frequency within a collection ¡Term postings within a document (a binary measure) ¡Term postings within a collection Indexing was taken from manually extracted keywords Results ¡Using the term frequency (or postings) within a collection always improved performance ¡Using term frequency ( or postings) within a document improved performance only for some collections 11

Other experiments involving ranking (cont. ) ¡ Harman(1986) Four term-weighting factors ¡(a) The number of matches between a document & a query ¡(b) The distribution of a term within a document collection IDF & noise measure ¡(c) The frequency of a term within a document ¡(d) The length of the document Results ¡Using the single measures alone, the distribution of the term within the collection = 2 (c) ¡Combining the within-document frequency with either the IDF or noise measure = 2 (using the IDF or noise alone) 12

Other experiments involving ranking (cont. ) ¡ Ranking based on document structure Not only using weights based on term importance both within an entire collection and within a given document (Bernstein and Williamson, 1984) But also using the structural position of the term ¡Summary versus text paragraphs In SIBRIS, increasing term-weights for terms in titles of documents and decreasing term-weights for terms added to a query from a thesaurus 13

Selecting ranking techniques ¡ Using term-weighting based on the distribution of a term within a collection always improves performance ¡ Within-document frequency + IDF weight often provides even more improvement ¡ Within-document frequency + (Several methods) IDF measure ¡ Adding additional weight for document structure Eg. higher weightings for terms appearing in the title or abstract vs. those appearing only in the text ¡ Relevance weighting (Chap 11) 14

The creation of an inverted file ¡ Implications for supporting inverted file structures Only the record id has to be stored (smaller index) Using strategies that increase recall at the expense of precision ¡ Inverted file is usually split into two pieces for searching The dictionary containing the term, along with statistics about that term such as no. of postings and IDF, and a pointer to the location of the postings file for term The postings file containing the record ids and the weights for all occurrences of the term 15

The creation of an inverted file (cont. ) ¡ 4 major options for storing weights in the postings file Store the raw frequency ¡ Slowest search ¡ Most flexible Store a normalized frequency ¡ Not suitable for use with the cosine similarity function ¡ Updating would not change the postings 16

The creation of an inverted file (cont. ) Store the completely weighted term ¡Any of the combination weighting schemes are suitable ¡Disadvantage: updating requires changing all postings If no within-record weighting is used, then the postings records do not have to store weights 17

Searching the inverted file ¡ Figure 14. 4 flowchart of search engine query parser Dictionary Lookup Dictionary entry Get Weights Record numbers on a per term basis Accumulator Record numbers. Total weights Sort by weight Ranked record numbers 18

Searching the inverted file (cont. ) ¡ Inefficiencies of this technique The I/O needs to be minimized ¡ A single read for all the postings of a given term, and then separating the buffer into record ids and weights Time savings can be gained at the expense of some memory space ¡Direct access to memory rather than through hashing A final major bottleneck can be the sort step of the “accumulators” for large data sets ¡Fast sort of thousands of records is very time consuming 19

Stemmed and unstemmed query terms ¡ If query terms were automatically stemmed in a ranking system, users generally got better results (Frakes, 1984; Canadela, 1990) In some cases, a stem is produced that leads to improper results ¡the original record terms are not stored in the inverted file; only their stems are used 20

Stemmed and unstemmed query terms (cont. ) ¡ Harman & Candela (1990) 2 separate inverted files could be created and stored ¡ Stem terms: normal query ¡ Unstemmed terms: don’t stem Hybrid inverted file ¡ Saving no space in the dictionary part ¡ Saving considerable storage (2 versions of posting) ¡ At the expense of some additional search time 21

A Boolean systems with ranking ¡ SIRE system Full Boolean capability + a variation of the basic search process ¡ Accepts queries that are either Boolean logic strings or natural language queries (implicit OR) ¡ Major modification to the basic search process Merge postings from the query terms before ranking is done ¡ Performance Faster response time for Boolean queries No increase in response time for natural language queries 22

Pruning ¡ A major time bottleneck in the basic search process The sort of the accumulators for large data sets ¡ Changed search algorithm with pruning: 1. Sort all query terms (stems) by decreasing IDF value 2. Do a binary search for the first term (i. e. , the highest IDF) and get the address of the postings list for that term 3. Read the entire postings file for that term into a buffer and add the term weights for each record id into the contents of the unique accumulator for the record id 23

Pruning (cont. ) 4. Check the IDF of the next query term. If the IDF >= 1/3 (max IDF of any term in the data set) then repeat steps 2, 3, and 4 otherwise repeat steps 2, 3, and 4, but do not add weights to zero weight accumulators 5. Sort the accumulators with nonzero weights to produce the final ranked record list 6. If a query has only high-frequency terms, then pruning cannot be done. 24

Thanks 25