Conjunction Dysfunction The Weakness of Conjunctive Queries in

Conjunction Dysfunction: The Weakness of Conjunctive Queries in Peer-to-Peer File-Sharing Systems 2006 IEEE P 2 P Conference, Cambridge, UK Wai Gen Yee, Linh Thai Nguyen, Ophir Frieder Information Retrieval Laboratory Illinois Institute of Technology IEEE P 2 P'06, Cambridge, UK

Objective n To improve the accuracy of queries in P 2 P file-sharing systems n More accurate queries q q More accurate queries reduce the time/resources required to find a desired file Reduces the number of impulse downloads IEEE P 2 P'06, Cambridge, UK 2

Impact n There are millions of daily users of P 2 P filesharing systems sharing petabytes of data IEEE P 2 P'06, Cambridge, UK 3

Claim n P 2 P file-sharing system query accuracy suffers from conjunctive query processing q Shared data descriptors are short n q Often filenames Long queries are too selective n Have an overall negative impact on accuracy IEEE P 2 P'06, Cambridge, UK 4

So Why Are Conjunctive Queries Implemented? n Likely used for the following reasons: q Result set precision increases with query length n q Better (? ) accuracy Result set size decreases with query length n Saves bandwidth IEEE P 2 P'06, Cambridge, UK 5

Proposed Solution n Use disjunctive query processing instead q Increase the likelihood that the desired result is contained in the result set (i. e. , query recall) n n q Query recall, not precision, is the problem Longer queries are not penalized Can be done in a cost-effective way IEEE P 2 P'06, Cambridge, UK 6

Related Work n Query specification is from commercial P 2 P file-sharing systems (e. g. , Gnutella) n IR community focuses on P 2 P sharing of text documents q n Recall is not a problem Networking and systems community focuses on routing q Queries and descriptors are single keys IEEE P 2 P'06, Cambridge, UK 7

Query Specification n n Peers are randomly connected Queries are flooded in the network Query IEEE P 2 P'06, Cambridge, UK 8

Query Processing n n n Users are looking for a particular file Servers share files Each file has a user-tuned descriptor q n Often, just the filename Conjunctive matching: Match occurs when all query terms are in the descriptor q q If Q D then return (D, serverid) to client Client uses D to decide if it wants to download the associated file IEEE P 2 P'06, Cambridge, UK 9

Example Query on e. Mule Query Descriptors Number of replicas IEEE P 2 P'06, Cambridge, UK “Key” 10

Conjunctive Query Example n Consider a query for Mozart’s Clarinet Concerto in A major by Michele Zukovsky candidate query terms mozart clarinet A major X X X zukovsky Num results 80 54 X 0 X 2 In general, longer queries return fewer results are returned. No results are returned using the most specific query IEEE P 2 P'06, Cambridge, UK 11

More on Query Processing n Results are grouped by key q n Generally an SHA-1 or MD 5 hash on the file Results are ranked by group size (the number of replicas) IEEE P 2 P'06, Cambridge, UK 12

Disjunctive and Conjunctive Query Processing n Disjunctive query processing q n If Q D≠ then return (D, serverid) to client Conjunctive query processing: q If Q D then return (D, serverid) to client IEEE P 2 P'06, Cambridge, UK 13

Experimental Setup n n 1, 000 peers 10, 000 queries q n n n Clients search for specific files Max descriptor size is 20 terms Average descriptor size is 6 terms ~20 files per peer at initialization Query length distribution: Length 1 2 3 4 5 6 7 8 Prob. . 28 . 30 . 18 . 13 . 05 . 03 . 02 . 01 IEEE P 2 P'06, Cambridge, UK 14

Data used in Simulation n Terms used to describe files are taken from Web documents q q q Each Web document corresponds to a file Random subset of 1, 000 files TREC WT 2 G IEEE P 2 P'06, Cambridge, UK 15

Metrics n Mean reciprocal rank q q Nq=number of total queries issued ranki=rank of desired result in result set n q q If the desired result is not in the result set, ranki= Measures query accuracy Appropriate when searching for a specific file IEEE P 2 P'06, Cambridge, UK 16

Metrics (cont’d) n Precision, Recall, F-score q q n A is the set of replicas of desired results R is the set of returned results F-score q Normalized combination of precision and recall IEEE P 2 P'06, Cambridge, UK 17

Overall Performance Difference With disjunction, MRR is 50% better. IEEE P 2 P'06, Cambridge, UK 18

Why Better Accuracy with Disjunction? Disjunctive queries return less precise results… But return desired results more enough. F-score increase indicates that the increase in recall compensates for the decrease in precision IEEE P 2 P'06, Cambridge, UK 19

MRR Requires Recall n In P 2 P file-sharing systems, results must be retrieved from servers before they are ranked q q A result set that does not contain the desired result cannot contribute to MRR Note: In traditional search engines, all results are stored at the server n Result set is potentially an ordered list of all server data IEEE P 2 P'06, Cambridge, UK 20

Source of the Performance Difference Disjunctive queries are not affected in this regard As queries grow, conjunctive matching is less likely to retrieve the desired result. disj conj IEEE P 2 P'06, Cambridge, UK 21

Consequently, Better MRR disj conj With disjunctive queries, MRR is more consistent over query length. IEEE P 2 P'06, Cambridge, UK 22

Cost Metric n Cost is measured in number of results returned to the client q n Measures network load and client processing Other metrics not considered q q User satisfaction Duration of search transaction IEEE P 2 P'06, Cambridge, UK 23

Controlling Cost n n Disjunctive queries increase cost Use result sampling to reduce cost IEEE P 2 P'06, Cambridge, UK 24

Result Sampling n n At the server, perform the following If Q D≠Ø then return (D, serverid) q n With probability P P is a user-defined sampling rate IEEE P 2 P'06, Cambridge, UK 25

Cost Reduction with Sampling disj conj At 25% sampling, the cost of disjunctive queries is lower than that of conjunctive queries (without sampling). Cost decreases at a predictable rate. IEEE P 2 P'06, Cambridge, UK 26

MRR Also Decreases with Sampling Drop in MRR Drop in %cont However, MRR with disjunctive queries and sampling is consistently better than MRR with conjunctive queries IEEE P 2 P'06, Cambridge, UK 27

Combining the Cost and Benefit… At 25% sampling, you yield… Higher accuracy Lower cost IEEE P 2 P'06, Cambridge, UK 28

Why Better Accuracy and Lower Cost? n If there are N≥ 1 instances of the desired result in the result set q q q Can sample away (N-1)/N * 100% of the result set We only need one instance of the desired result in the result set (N-1)/N approaches 1 quickly IEEE P 2 P'06, Cambridge, UK 29

Conclusion n In P 2 P file-sharing applications, conjunctive queries may be too selective q n n Descriptors are too short Disjunctive queries improve accuracy by improving recall Cost can be controlled by sampling IEEE P 2 P'06, Cambridge, UK 30

Ongoing Work n n n Dynamic determination of a sampling rate The use of different result ranking techniques Combining conjunction/disjunction in query processing IEEE P 2 P'06, Cambridge, UK 31

Questions? n Wai Gen Yee (yee@iit. edu) Linh Thai Nguyen (nguylin@iit. edu) Ophir Frieder (ophir@ir. iit. edu) n Information Retrieval Lab n n q q Ilinois Institute of Technology ir. iit. edu IEEE P 2 P'06, Cambridge, UK 32