Introduction to IR Research Cheng Xiang Zhai Department

Introduction to IR Research Cheng. Xiang Zhai Department of Computer Science Graduate School of Library & Information Science Institute for Genomic Biology, Statistics University of Illinois, Urbana-Champaign http: //www-faculty. cs. uiuc. edu/~czhai, czhai@cs. uiuc. edu 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 1

What is Research? • • Research – Discover new knowledge – Seek answers to questions Basic research – Goal: Expand man’s knowledge (e. g. , which genes control social behavior of honey bees? ) – Often driven by curiosity (but not always) – High impact examples: relativity theory, DNA, … Applied research – Goal: Improve human condition (i. e. , improve the wolrd) (e. g. , how to cure cancers? ) – Driven by practical needs – High impact examples: computers, transistors, vaccinations, … The boundary is vague; distinction isn’t important 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 2

Why Research? Curiosity Amount of knowledge Basic Research Utility of Advancement of Applications Technology Applied Research 2008 © Cheng. Xiang Zhai Quality of Life Application Development Dragon Star Lecture at Beijing University, June 21 -30, 2008 3

Where’s IR Research? Utility of Advancement of Applications Technology Amount of knowledge Quality of Life Computer Science Info rma tion Basic Research Application Sci enc e Development Applied Research 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 4

Where’s Your Position? Different position benefits from different collaborators Amount of knowledge Basic Research Utility of Advancement of Applications Technology Applied Research 2008 © Cheng. Xiang Zhai Quality of Life Application Development Dragon Star Lecture at Beijing University, June 21 -30, 2008 5

Research Process • Identification of the topic (e. g. , Web search) • Hypothesis formulation (e. g. , algorithm X is better than Y=state-of-the-art) • Experiment design (measures, data, etc) (e. g. , retrieval accuracy on a sample of web data) • Test hypothesis (e. g. , compare X and Y on the data) • Draw conclusions and repeat the cycle of hypothesis formulation and testing if necessary (e. g. , Y is better only for some queries, now what? ) 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 6

Typical IR Research Process • • • Look for a high-impact topic (basic or applied) New problem: define/frame the problem Identify weakness of existing solutions if any Propose new methods Choose data sets (often a main challenge) Design evaluation measures (can be very difficult) Run many experiments (need to have clear research hypotheses) Analyze results and repeat the steps above if necessary Publish research results 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 7

Research Methods • Exploratory research: Identify and frame a new problem (e. g. , “a survey/outlook of personalized search”) • Constructive research: Construct a (new) solution to a problem (e. g. , “a new method for expert finding”) • Empirical research: evaluate and compare existing solutions (e. g. , “a comparative evaluation of link analysis methods for web search”) • The “E-C-E cycle”: exploratory constructive empirical exploratory… 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 8

Types of Research Questions and Results • Exploratory (Framework): What’s out there? • Descriptive (Principles): What does it look like? How does it work? • Evaluative (Empirical results): How well does a method solve a problem? • Explanatory (Causes): Why does something happen the way it happens? • Predictive (Models): What would happen if xxx ? 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 9

Look for New IR Research Questions • • • Driven by new data: X is a new type of data emerging (e. g. , X= blog vs. news) – How is X different from existing types of data? – What new issues/problems are raised by X? – Are existing methods sufficient for solving old problems on X? If not, what are the new challenges? – What new methods are needed? – Are old evaluation measures adequate? Driven by new users: Y is a set of new users (e. g. , ordinary people vs. librarians) – How are the new users different from old ones? What new needs do they have? – Can existing methods work well to satisfy their needs? If not, what are the new challenges? – What new functions are appropriate for Y? Driven by new tasks (not necessarily new users or new data): Z is a new task (e. g. , social networking, online shopping) – What information management functions are needed to better support Z? – Can these new functions reduced to old ones? If not, what are the new challenges? 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 10

Solid and High Impact Research • Solid work: – A clear hypothesis (research question) with conclusive result (either positive or negative) – Clearly adds to our knowledge base (what can we learn from this work? ) • – Implications: a solid, focused contribution is often better than a non-conclusive broad explorations High impact = high-importance-of-problem * high-quality-ofsolution – high impact = open up an important problem – high impact = close a problem with the best solution – high impact = major milestones in between – Implications: question the importance of the problem and don’t just be satisfied with a good solution, make it the best 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 11

Where to Publish IR Papers • • • Core IR conferences: – ACM SIGIR, ACM CIKM – ECIR, AIRS Core IR journals – ACM TOIS, IRJ – IPM, JASIS Web Applications – WWW, WSDM Other related conferences – – Natural Language Processing: HLT, ACL, NAACL, COLING, EMNLP Machine Learning: ICML, NIPS Data Mining: KDD, ICDM Databases: SIGMOD, VLDB, ICDE … 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 12

Your Assignment • Write a team research proposal in response to the “course request for proposals (RFP)” • The 5 -5 rules: – 5 pages (single column, 10 points or 11 points font) – Up to 5 team members – At least 5 references – 5 components (to be described on the next slide) – Due 5 days after the lecture period • 2 -page outline to be finished within the lecture period 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 13

5 Elements of a Research Proposal • • • Problem statement – Motivation: Raise the research question – Define the question – Benefit of solving the problem Previous/Related work – What do we already know about the problem? – How would the proposed research increase our knowledge? Research plan – How do you plan to solve the problem? – Justify your plan (analytical, preliminary results) – Evaluation plan Research management – Schedule, milestones, fallback plans Resource needs 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 14

Next Lecture: Prepare Yourself to do IR research 2008 © Cheng. Xiang Zhai Dragon Star Lecture at Beijing University, June 21 -30, 2008 15