ContextAware Ranking in Web Search SIGIR 10 Biao

Context-Aware Ranking in Web Search (SIGIR 10’) Biao Xiang, Daxin Jiang, Jian Pei, Xiaohui Sun, Enhong Chen, Hang Li 2010/10/26 1

Outline l Introduction l Ranking Principles ¡ Context-aware Ranking Principles ¡ Effectiveness of Principles l Context-Aware Ranking l Experimental Results l Conclusions 2

Introduction l almost all the existing ranking models consider only the current query and the documents l do not take into account any context information ¡ the previous queries in the same session ¡ the answers clicked on ¡ skipped by the user to the previous queries 3

l two critical problems about context-aware ranking for Web search ¡ ¡ How can we take advantage of different types of contexts in ranking? How can we integrate context information into a ranking model? discuss the types of contexts and propose ranking principles l evaluate the effectiveness of the principles l incorporate context information into a learning-to -rank model l 4

Ranking Principles *Context-Aware Ranking Principles l Reformulation C S l Principle 1 (Reformulation). For consecutive queries in a session such that qt reformulates , if a search result d for is clicked on or skipped, as a result for is unlikely to be clicked on and thus should be demoted. 5

l Specialization l Principle 2 (Specialization). For consecutive queries in a session such that qt specializes , the user likely prefers the search results specifically focusing on. 6

l Generalization l Principle 3 (Generalization). For consecutive queries in a session such that generalizes , the user would likely not prefer the search results specifically focusing on qt− 1. 7

l General Association l Principle 4 (General association). For consecutive queries in a session such that and are generally associated, the user likely prefers the search results related to both and qt. Such results should be promoted for qt. 8

How to import the principles l Principle 2 ¡ : the set of terms appearing in query but not in query. ¡ promote the results matching in the set of answers to. l Principle 3 ¡ : the set of terms appearing in query but not in query. ¡ promote the results matching in the set of answers to. 9

l Principle 4 ¡ choose any topic taxonomy such as the Open Directory Project (ODP) ¡ ( ): the sets of topics of ( ) ¡ : the set of common topics between and ¡ promote a search result u if the set of topics of u shares at least one topic with. 10

Ranking Principles *Effectiveness of Principles l 37, 320 user sessions l successive query pairs within the same sessions l manually labeled the relations l 10, 000 randomly selected successive query pairs. 11

la search result satisfies the principle if should be promoted(2, 4) or not(1, 3) l : the set of search results that satisfy the principle l : consists of the search results that were clicked on by the users l estimated Δ = P(c = 1|h = 1)− P(c = 1|h =0) 12

l Evaluation ¡ in Different Types of Contexts the two successive queries match the relation of the principle l Evaluation in All Contexts 13

Context-Aware Ranking l Rank. SVM : an SVM model for classification on the preference between a pair of document. the original ranking list from the search engine the list from the Rank. SVM-R 0 Rank. SVM-R 1 Rank. SVM-F re- ranking 14

Experimental Results 1500 cases, 1000 for training, 500 for validation l , l 15

Conclusions human labeled data and user click data complementary to each other l the four context-aware methods are better than the search engine l ¡ l all three Rank. SVM methods perform better than the baseline in context-aware ranking. ¡ l the effectiveness of contextaware ranking. consider different types of contexts in Web search. the Rank. SVM-F and Rank. SVM-R 1 methods show larger improvements than Rank. SVM-R 0 ¡ the usefulness of considering the original ranking of the search engine. 16