Excuse me what are you searching for Response

Excuse me, what are you searching for? Response quality refinement in content-based information retrieval Masaryk University, Faculty of Informatics 1/14 Petra Budíková

Outline § § Content-Based Information Retrieval § Reasons, approaches, difficulties Relevance feedback: § Basic principle, advantages § Relevance feedback in CBIR: state-of-the-art, specific problems § § MUFIN: Real-world application § Web-scale similarity searching in images Our work: efficient response refinement § Relevance feedback implementation in MUFIN § Reordering, overlap prediction, multi-object query 2/14

Content-based Information Retrieval § Motivation: § Traditional DB technologies no longer sufficient for many new data types (images, sounds, …) § In huge amounts of data user searches rather for similarity than for exact match § Metric space approach § § Universal model Data objects + black-box distance function Object described by several low-level features Query by example paradigm 3/14

Difficulties § Semantic gap between human understanding of similarity (high-level) and low-level descriptors In a nutshell, “an image is worth a thousand words”, and the machine does not know what these words are. § Subjectivity of understanding similarity § Similarity context-dependent Universal measure of similarity does not exist 4/14

Relevance feedback § § Bridging the semantic gap Getting the user into the search process Step 1. Machine provides an initial retrieval results, through query-by-keyword, sketch, or example, etc. Step 2. User provides a judgment on the currently displayed items as to whether, and to what degree, they are relevant or irrelevant to her/his request. Step 3. Machine learns and tries again. Go to step 2. 5/14

Relevance feedback in CBIR § Initial query § Query object + distance function § Returns k nearest objects § Learning – basic principles: § Based on the feedback, adjust the query object § e. g. weighted average of positive examples § or, adjust the distance function § enhance the features common to positive examples 6/14

Relevance feedback - challenges § § Effectivity: learn the right thing § (In)Consistency of user evaluation § Small number of samples § Positive/negative examples treatment § … Efficiency: learn and interact quickly § The user is supposed to be Greedy and Impatient § Only a few iterations acceptable § Each iteration should be evaluated quickly 7/14

MUFIN: Real-world application § MUFIN: MUlti-Feature Indexing Network § General system for similarity search § Distributed P 2 P architecture → scalable § MUFIN Image search: § 100 million images from Flickr § Five visual descriptors (MPEG 7 color, texture) § Interactive web interface § http: //mufin. fi. muni. cz/imgsearch/ 8/14

MUFIN Image Search Submit query to the network Choose promising peers, evaluate query, get results, put them together, show answer + distance function Get feedback, improve query object / distance function 9/14

Efficiency § § Our goal: interactive searching in very large databases Relevance feedback issues additional accesses to the search engine § Expensive! 300 ms – 1 s § § Choose promising peers, evaluate query, get results, put them together, show answer May result in only slightly different answer set Avoid unnecessary new queries! 10/14

Reordering § In the feedback loop, do not access the search engine at all § For the same price, we can get more than k objects during the initial query evaluation § Instead of 20, get 2000 objects § § Reorder these objects using the feedback information May improve the result with minimal costs § § Works well if there are enough good objects in the working set Easy implementation vs. no guarantees on quality 11/14

RF with overlap prediction § Feedback loop with new access to the search engine § Access only nodes that have not been visited before Questions: § § § What shall be remembered, what visited in the feedback loop and what skipped? Can observations of trend changes be helpful? 12/14

Multi-object query § RF usually modifies query object § § § Searching more difficult with modified distance function Modified object near to the original one § Slow progress Alternative approach: multi-object query § § Evaluate a query for each of the objects in parallel Combine the results § Sum the distances, positions, … § Handle overlaps efficiently 13/14

Conclusion § § To get useful content-based search system, some type of interaction with user is essential to capture his/her needs Relevance feedback methods provide a good potential, however the speed of evaluation must be taken into consideration Make the most of all information that can be easily (quickly) gained during the retrieval, economize on the search engine accesses 14/14